A communications equipment manufacturing company has a product which is responsible for emitting informative signals. Company wants to build a machine learning model which can help the company to predict the equipment’s signal quality using various parameters.
The data set contains information on various signal tests performed:
The need is to build a classifier which can use these parameters to determine the signal strength or quality as number.
Liveplot.import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib.style as style; style.use('fivethirtyeight')
from IPython.display import clear_output,Markdown
%matplotlib inline
plt.rcParams["figure.figsize"] = (15,10)
pd.set_option('display.max_colwidth', 500)
from sklearn.metrics import roc_auc_score, mean_absolute_error, r2_score
from sklearn.preprocessing import StandardScaler,MinMaxScaler
from sklearn.model_selection import train_test_split
import sys
from itertools import permutations
import tensorflow as tf
from tensorflow.keras.layers import Input, Dense, BatchNormalization, Activation, Dropout
from tensorflow.keras.models import Model
from tensorflow.keras.initializers import RandomNormal,Constant
import optuna
### Only used if Google Colab is used.
# from google.colab import drive
# drive.mount('/content/drive')
df = pd.read_csv("Part- 1,2&3 - Signal.csv")
df.head()
| Parameter 1 | Parameter 2 | Parameter 3 | Parameter 4 | Parameter 5 | Parameter 6 | Parameter 7 | Parameter 8 | Parameter 9 | Parameter 10 | Parameter 11 | Signal_Strength | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 7.4 | 0.70 | 0.00 | 1.9 | 0.076 | 11.0 | 34.0 | 0.9978 | 3.51 | 0.56 | 9.4 | 5 |
| 1 | 7.8 | 0.88 | 0.00 | 2.6 | 0.098 | 25.0 | 67.0 | 0.9968 | 3.20 | 0.68 | 9.8 | 5 |
| 2 | 7.8 | 0.76 | 0.04 | 2.3 | 0.092 | 15.0 | 54.0 | 0.9970 | 3.26 | 0.65 | 9.8 | 5 |
| 3 | 11.2 | 0.28 | 0.56 | 1.9 | 0.075 | 17.0 | 60.0 | 0.9980 | 3.16 | 0.58 | 9.8 | 6 |
| 4 | 7.4 | 0.70 | 0.00 | 1.9 | 0.076 | 11.0 | 34.0 | 0.9978 | 3.51 | 0.56 | 9.4 | 5 |
Parameter 1 to Parameter 11 are independent variables.Signal Strength is the independent variable.df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 1599 entries, 0 to 1598 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Parameter 1 1599 non-null float64 1 Parameter 2 1599 non-null float64 2 Parameter 3 1599 non-null float64 3 Parameter 4 1599 non-null float64 4 Parameter 5 1599 non-null float64 5 Parameter 6 1599 non-null float64 6 Parameter 7 1599 non-null float64 7 Parameter 8 1599 non-null float64 8 Parameter 9 1599 non-null float64 9 Parameter 10 1599 non-null float64 10 Parameter 11 1599 non-null float64 11 Signal_Strength 1599 non-null int64 dtypes: float64(11), int64(1) memory usage: 150.0 KB
Parameter 1 to Parameter 11 are of data type float.Signal Strength is of datatype integerdtemp = pd.DataFrame(df.describe().T)
dtemp['dtypes'] = df.dtypes
dtemp['N-Uniques'] = df.nunique()
dtemp
| count | mean | std | min | 25% | 50% | 75% | max | dtypes | N-Uniques | |
|---|---|---|---|---|---|---|---|---|---|---|
| Parameter 1 | 1599.0 | 8.319637 | 1.741096 | 4.60000 | 7.1000 | 7.90000 | 9.200000 | 15.90000 | float64 | 96 |
| Parameter 2 | 1599.0 | 0.527821 | 0.179060 | 0.12000 | 0.3900 | 0.52000 | 0.640000 | 1.58000 | float64 | 143 |
| Parameter 3 | 1599.0 | 0.270976 | 0.194801 | 0.00000 | 0.0900 | 0.26000 | 0.420000 | 1.00000 | float64 | 80 |
| Parameter 4 | 1599.0 | 2.538806 | 1.409928 | 0.90000 | 1.9000 | 2.20000 | 2.600000 | 15.50000 | float64 | 91 |
| Parameter 5 | 1599.0 | 0.087467 | 0.047065 | 0.01200 | 0.0700 | 0.07900 | 0.090000 | 0.61100 | float64 | 153 |
| Parameter 6 | 1599.0 | 15.874922 | 10.460157 | 1.00000 | 7.0000 | 14.00000 | 21.000000 | 72.00000 | float64 | 60 |
| Parameter 7 | 1599.0 | 46.467792 | 32.895324 | 6.00000 | 22.0000 | 38.00000 | 62.000000 | 289.00000 | float64 | 144 |
| Parameter 8 | 1599.0 | 0.996747 | 0.001887 | 0.99007 | 0.9956 | 0.99675 | 0.997835 | 1.00369 | float64 | 436 |
| Parameter 9 | 1599.0 | 3.311113 | 0.154386 | 2.74000 | 3.2100 | 3.31000 | 3.400000 | 4.01000 | float64 | 89 |
| Parameter 10 | 1599.0 | 0.658149 | 0.169507 | 0.33000 | 0.5500 | 0.62000 | 0.730000 | 2.00000 | float64 | 96 |
| Parameter 11 | 1599.0 | 10.422983 | 1.065668 | 8.40000 | 9.5000 | 10.20000 | 11.100000 | 14.90000 | float64 | 65 |
| Signal_Strength | 1599.0 | 5.636023 | 0.807569 | 3.00000 | 5.0000 | 6.00000 | 6.000000 | 8.00000 | int64 | 6 |
Parameter 8 has a very short range of 0.001 and may not add significant variance to the overall data set.Parameter 5 has a very small interquartile range and with some significant outliers to the right of the curve.Signal Strength is a discrete integer between 3 to 8.df.isnull().sum() , df.isna().sum()
(Parameter 1 0 Parameter 2 0 Parameter 3 0 Parameter 4 0 Parameter 5 0 Parameter 6 0 Parameter 7 0 Parameter 8 0 Parameter 9 0 Parameter 10 0 Parameter 11 0 Signal_Strength 0 dtype: int64, Parameter 1 0 Parameter 2 0 Parameter 3 0 Parameter 4 0 Parameter 5 0 Parameter 6 0 Parameter 7 0 Parameter 8 0 Parameter 9 0 Parameter 10 0 Parameter 11 0 Signal_Strength 0 dtype: int64)
NAN or NULL in the given dataframe. No additional cleansing required.# Assign the target variable
target = 'Signal_Strength'
%run edafunc.py
cat_var_list,cont_var_list,cats_df1= create_variable_list(df,10,stats=True)
cats_df1
| Variable | Type | Categorical_Class | Uniques | N-Uniques | |
|---|---|---|---|---|---|
| 0 | Signal_Strength | Categorical | Multi | [5, 6, 7, 4, 8, 3] | 6 |
| 1 | Parameter 1 | Numeric | NaN | NaN | 96 |
| 2 | Parameter 2 | Numeric | NaN | NaN | 143 |
| 3 | Parameter 3 | Numeric | NaN | NaN | 80 |
| 4 | Parameter 4 | Numeric | NaN | NaN | 91 |
| 5 | Parameter 5 | Numeric | NaN | NaN | 153 |
| 6 | Parameter 6 | Numeric | NaN | NaN | 60 |
| 7 | Parameter 7 | Numeric | NaN | NaN | 144 |
| 8 | Parameter 8 | Numeric | NaN | NaN | 436 |
| 9 | Parameter 9 | Numeric | NaN | NaN | 89 |
| 10 | Parameter 10 | Numeric | NaN | NaN | 96 |
| 11 | Parameter 11 | Numeric | NaN | NaN | 65 |
sncp_c1 = sns.color_palette("muted", 40)
num_cols=4
display(Markdown("# <center><u>Univariate Analysis</u></center>\n\n"))
for color,index in enumerate((range(0,len(cont_var_list)))):
fig, ax = plt.subplots(1,num_cols,figsize=(20,4))
display(Markdown(f"## <center>`{cont_var_list[index].upper()}`</center></br>"))
sns.boxplot(x=df[cont_var_list[index]],color=sncp_c1[color],ax=ax[0]);
ax[0].set_title(f'Boxplot with Outliers \n',{'fontsize':15})
sns.histplot(x=df[cont_var_list[index]],kde=True,color=sncp_c1[color],ax=ax[1]);
ax[1].set_title(f'Histogram with Outliers \n',{'fontsize':15})
ol_temp = sp.outliers_iqr(df[cont_var_list[index]])
sns.boxplot(x=ol_temp,color=sncp_c1[color],ax=ax[2]);
ax[2].set_title(f'Boxplot Outliers removed\n',{'fontsize':15})
sns.histplot(x=ol_temp,color=sncp_c1[color],ax=ax[3],kde=True);
ax[3].set_title(f'Histogram Outliers removed\n',{'fontsize':15})
plt.show()
Parameter 5 and Parameter 8 have very short IQR.# Heatmap of of all variables.
display(Markdown(f"## <center>`Heatmap of correlations between all features`</center></br>"))
fig,ax = plt.subplots(figsize=(20,10))
sns.heatmap(df.corr(),annot=True);
display(Markdown(f"## <center>`Pair Plots`</center></br>"))
sns.pairplot(data=df,diag_kind='kde',hue='Signal_Strength');
1.There are significant correlations between most of the variables.
2.Most significant correlations are between Parameter 1 and Parameter 3, Parameter 1 and Parameter 8, Parameter 6 and Parameter 7 at almost 0.67.
3.Signal Strength is highly correlated with Parameter 10 and Parameter 11
def show_variance_table(df,target,threshold=0.0):
from sklearn.feature_selection import VarianceThreshold
columns_to_show = df.drop(target,axis=1).columns
vthreshold = VarianceThreshold(threshold=threshold)
vthreshold.fit(df[columns_to_show].values)
vt_df = pd.DataFrame({'Feature':columns_to_show,
'Variance Threshold':vthreshold.variances_}
).sort_values(by='Variance Threshold',ascending=False)
return vt_df
%run edafunc.py
show_variance_threshold(df,target)
display(show_variance_table(df,target,))
| Feature | Variance Threshold | |
|---|---|---|
| 6 | Parameter 7 | 283.000000 |
| 5 | Parameter 6 | 71.000000 |
| 0 | Parameter 1 | 3.029521 |
| 3 | Parameter 4 | 1.986654 |
| 10 | Parameter 11 | 1.134937 |
| 2 | Parameter 3 | 0.037924 |
| 1 | Parameter 2 | 0.032042 |
| 9 | Parameter 10 | 0.028715 |
| 8 | Parameter 9 | 0.023820 |
| 4 | Parameter 5 | 0.002214 |
| 7 | Parameter 8 | 0.000004 |
Parameter 1, Parameter 4, Parameter 6, Parameter 7 and Parameter 11 are the top 5 variables showing high variance in the data.Parameter 8 shows nearly zero variance with Parameter 5 the next to least. These parameters could be discarded during the modellingdef get_VIF_Table(df,target):
from statsmodels.stats.outliers_influence import variance_inflation_factor
df = df.drop(target,axis=1).copy()
vif_data = pd.DataFrame()
vif_data["Feature"] = df.columns
vif_data["VIF"] =[variance_inflation_factor(df.values, i) for i in range(len(df.columns))]
return vif_data.sort_values(by='VIF',ascending=False)
get_VIF_Table(df,target)
| Feature | VIF | |
|---|---|---|
| 7 | Parameter 8 | 1479.287209 |
| 8 | Parameter 9 | 1070.967685 |
| 10 | Parameter 11 | 124.394866 |
| 0 | Parameter 1 | 74.452265 |
| 9 | Parameter 10 | 21.590621 |
| 1 | Parameter 2 | 17.060026 |
| 2 | Parameter 3 | 9.183495 |
| 4 | Parameter 5 | 6.554877 |
| 6 | Parameter 7 | 6.519699 |
| 5 | Parameter 6 | 6.442682 |
| 3 | Parameter 4 | 4.662992 |
Parameter 8, 9, 11,1,10 and 2 have very high VIFs i.e multi-collienarity.class DNN_Model() :
def __init__(self,type='regression'):
self.type = type
def create_model(self,in_shape,outs,**kwargs):
self.in_shape = in_shape
if kwargs != {}:
self.num_layers = kwargs['num_layers'] if 'num_layers' in kwargs else 1
self.batch_size = kwargs['batch_size'] if 'batch_size' in kwargs else 10
self.activation = kwargs['activation'] if 'activation' in kwargs else 'relu'
self.kernel_init = kwargs['kernel_init'] if 'kernel_init' in kwargs else 'uniform'
self.optimizer = kwargs['opts'] if 'opts' in kwargs else tf.keras.optimizers.Adam(learning_rate=0.001)
self.neurons_per_layer = kwargs['neurons_per_layer'] if 'neurons_per_layer' in kwargs else [64]*self.num_layers
self.drop_rate_per_layer = kwargs['drop_rate_per_layer'] if 'drop_rate_per_layer' in kwargs else [0.2]*self.num_layers
self.n_epochs = kwargs['n_epochs'] if 'n_epochs' in kwargs else 50
self.arch = kwargs['arch'] if 'arch' in kwargs else 'B A D'
self.momentum = kwargs['momentum'] if 'meomentum' in kwargs else 0.99
self.epsilon = kwargs['epsilon'] if 'epsilon' in kwargs else 0.005
self.beta_init_std = kwargs['beta_init_std'] if 'beta_init_std' in kwargs else 0.05
self.gamma_init = kwargs['gamma_init'] if 'gamma_init' in kwargs else 0.9
self.center = kwargs['center'] if 'center' in kwargs else True
self.scale = kwargs['scale'] if 'scale' in kwargs else False
else:
self.num_layers = 1
self.batch_size = 10
self.activation = 'relu'
self.kernel_init = 'normal'
self.optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
self.neurons_per_layer = [32] * self.num_layers
self.drop_rate_per_layer = [0.2] * self.num_layers
self.n_epochs = 50
self.arch = 'B A D'
self.momentum = 0.99
self.epsilon = 0.005
self.beta_init_std = 0.05
self.gamma_init = 0.9
self.center = True
self.scale = False
if self.type == 'regression':
self.op_activation = 'linear'
self.outs = 1
elif self.type == 'classification':
self.op_activation = 'softmax'
self.outs = outs
inputs = Input(shape=self.in_shape)
for layer in range(self.num_layers):
if layer == 0 :
m = Dense(self.neurons_per_layer[layer],kernel_initializer = self.kernel_init)(inputs)
else:
m = Dense(self.neurons_per_layer[layer],kernel_initializer = self.kernel_init)(m)
for keys in self.arch.split(' '):
if keys == 'A':
m = Activation(self.activation)(m)
elif keys == 'B':
m = BatchNormalization(momentum = self.momentum,
epsilon = self.epsilon,
beta_initializer = RandomNormal(mean=0.0, stddev=self.beta_init_std),
gamma_initializer = Constant(value=self.gamma_init),
center = self.center,
scale = self.scale)(m)
elif keys == 'D':
m = Dropout(self.drop_rate_per_layer[layer])(m)
outputs = Dense(self.outs, activation=self.op_activation,kernel_initializer = self.kernel_init)(m)
model = Model(inputs=inputs, outputs=outputs)
return(model)
def model_summary(self,model):
return model.summary()
def train_model(self,model,X_train,y_train,loss,metric,callbacks=None,validation_split=0.2,validation_data=None,verbose=1):
self.model = model
self.loss = loss
self.metric = metric
self.model.compile(loss=self.loss,
optimizer=self.optimizer,
metrics=self.metric)
if callbacks == None:
callbacks = []
else :
callbacks = callbacks
if validation_data == None:
self.history = self.model.fit(X_train,
y_train,
epochs=self.n_epochs,
validation_split=validation_split,
callbacks=callbacks,
verbose=0)
else:
self.history = self.model.fit(X_train,
y_train,
epochs=self.n_epochs,
validation_data=validation_data,
callbacks=callbacks,
verbose=0)
return
def test_model(self,X_test,y_test,callbacks=None):
self.test_results = self.model.evaluate(X_test,y_test,batch_size=self.batch_size,callbacks=callbacks,verbose=0)
return self.test_results
class LivePlot(tf.keras.callbacks.Callback):
def __init__(self,refresh_rate=5,train_loss=None,train_metric=None):
self.validation_prefix = "val_"
self.refresh_rate = refresh_rate
self.train_loss = train_loss
self.val_loss = self.validation_prefix + train_loss
self.train_metric = train_metric
self.val_metric = self.validation_prefix + train_metric
# This function is called when the training begins
def on_train_begin(self, logs={}):
# Initialize the lists for holding the logs, losses and metrics
self.train_losses = []
self.train_metrics = []
self.val_losses = []
self.val_metrics = []
self.logs = []
# This function is called at the end of each epoch
def on_epoch_end(self, epoch, logs={}):
"""
Calculates and plots loss and metrics
"""
# Extract from the log
log_train_loss = logs.get(self.train_loss)
log_train_metric = logs.get(self.train_metric)
log_val_loss = logs.get(self.val_loss)
log_val_metric = logs.get(self.val_metric)
# Append the logs, losses and accuracies to the lists
self.logs.append(logs)
self.train_losses.append(log_train_loss)
self.train_metrics.append(log_train_metric)
self.val_losses.append(log_val_loss)
self.val_metrics.append(log_val_metric)
# Plots every nth epoch
if epoch > 0 and epoch%self.refresh_rate == 0:
fig, ax = plt.subplots(1,2,figsize=(20,6))
clear_output(wait=True)
N = np.arange(0, len(self.train_losses))
sns.lineplot(x=N,y=self.train_losses,ax=ax[0],legend='brief',label=self.train_loss)
sns.lineplot(x=N, y = self.val_losses,ax=ax[0],legend='brief',label=self.val_loss)
ax[0].set_title(f'Loss over Epoch {epoch} - {log_train_loss:.3}/{log_val_loss:.3} - (T/V)\n',{'fontsize':20})
ax[0].set_xlabel('Epochs')
ax[0].set_ylabel(self.train_loss)
sns.lineplot(x=N,y=self.train_metrics,ax=ax[1],legend='brief',label=self.train_metric)
sns.lineplot(x=N,y=self.val_metrics,ax=ax[1],legend='brief',label=self.val_metric)
ax[1].set_title(f'Performance over Epoch {epoch} - {log_train_metric:.3}/{log_val_metric:.3} - (T/V) \n',{'fontsize':20})
ax[1].set_xlabel('Epochs')
ax[1].set_ylabel(self.train_metric)
plt.show()
baseline model/data will then be subjected to different levels of hyperparameter optimization.baseline the model, we will assume some basic hyperparameters as show below.Loss and Performance metric for regression problem will spare_categorical_crossentropy and accuracy respectively.columns_to_drop = [target]
X = df.drop(columns_to_drop,axis=1)
y = df[target]
n_class = len(np.unique(y))
print(f" Number of output classes : {n_class}")
print(f"Before --- Min:{np.min(y)} and Max:{np.max(y)}")
y = y-3
print(f"After --- Min:{np.min(y)} and Max:{np.max(y)}")
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
X_train.shape, X_test.shape
Number of output classes : 6 Before --- Min:3 and Max:8 After --- Min:0 and Max:5
((1279, 11), (320, 11))
## Baseline Params
kwargs = {'num_layers': 1,
'arch' :'B A D',
# 'neurons_per_layer':[64],
# 'batch_size': 10,
'drop_rate_per_layer': [0.],
# 'activation': 'relu',
'n_epochs': 100,
# 'kernel_init': 'uniform',
# 'lr': 0.001,
# 'opts': 'adam'
}
loss ='sparse_categorical_crossentropy'
metric ='accuracy'
score_board = pd.DataFrame(columns=['Baseline','Loss','Metric','Notes'])
dnn_1 = DNN_Model('classification')
m_1 = dnn_1.create_model(X_train.shape[1],n_class)
display(Markdown("### Base Model Summary"))
m_1.summary()
Model: "model_8" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_9 (InputLayer) [(None, 4)] 0 _________________________________________________________________ dense_21 (Dense) (None, 32) 160 _________________________________________________________________ batch_normalization_13 (Batc (None, 32) 96 _________________________________________________________________ activation_13 (Activation) (None, 32) 0 _________________________________________________________________ dropout_13 (Dropout) (None, 32) 0 _________________________________________________________________ dense_22 (Dense) (None, 6) 198 ================================================================= Total params: 454 Trainable params: 390 Non-trainable params: 64 _________________________________________________________________
dnn_1.__dict__
{'type': 'classification',
'in_shape': 4,
'num_layers': 1,
'batch_size': 10,
'activation': 'relu',
'kernel_init': 'normal',
'optimizer': <tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7ff2882b0070>,
'neurons_per_layer': [32],
'drop_rate_per_layer': [0.2],
'n_epochs': 50,
'arch': 'B A D',
'momentum': 0.99,
'epsilon': 0.005,
'beta_init_std': 0.05,
'gamma_init': 0.9,
'center': True,
'scale': False,
'op_activation': 'softmax',
'outs': 6}
batchNormalization, Activation and Dropout layers in that order.dnn_1 = DNN_Model('classification')
m_1 = dnn_1.create_model(X_train.shape[1],n_class,**kwargs)
callbacks = LivePlot(5,train_loss='loss',train_metric='accuracy')
# callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
display(Markdown(f"### Test Loss : {result[0]} -- Test Accuracy : {result[1]}"))
score_board = score_board.append({'Baseline':'No feature engineering/selection',
'Loss':result[0],
'Metric':result[1],
'Notes':'Loss: Sparse categorical entropy -- Metric : Accuracy '},ignore_index=True)
score_board
| Baseline | Loss | Metric | Notes | |
|---|---|---|---|---|
| 0 | No feature engineering/selection | 0.958858 | 0.621875 | Loss: Sparse categorical entropy -- Metric : Accuracy |
We will change the original variables into quadratic variables `(degree 2)` and drop `Parameter 8` as it's very close to `1` and increasing the power will not create any significant variance.
dff = df.copy()
dff['Parameter 1'] = (dff['Parameter 1'])**2
dff['Parameter 2'] = (dff['Parameter 2'])**2
dff['Parameter 3'] = (dff['Parameter 3'])**2
dff['Parameter 4'] = (dff['Parameter 4'])**2
dff['Parameter 5'] = (dff['Parameter 5'])**2
dff['Parameter 6'] = (dff['Parameter 6'])**2
dff['Parameter 7'] = (dff['Parameter 7'])**2
dff['Parameter 8'] = (dff['Parameter 8'])**2
dff['Parameter 9'] = (dff['Parameter 9'])**2
dff['Parameter 10'] = (dff['Parameter 10'])**2
dff['Parameter 11'] = (dff['Parameter 11'])**2
dff.drop(['Parameter 8'],axis=1,inplace=True)
dff.head()
| Parameter 1 | Parameter 2 | Parameter 3 | Parameter 4 | Parameter 5 | Parameter 6 | Parameter 7 | Parameter 9 | Parameter 10 | Parameter 11 | Signal_Strength | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 54.76 | 0.4900 | 0.0000 | 3.61 | 0.005776 | 121.0 | 1156.0 | 12.3201 | 0.3136 | 88.36 | 5 |
| 1 | 60.84 | 0.7744 | 0.0000 | 6.76 | 0.009604 | 625.0 | 4489.0 | 10.2400 | 0.4624 | 96.04 | 5 |
| 2 | 60.84 | 0.5776 | 0.0016 | 5.29 | 0.008464 | 225.0 | 2916.0 | 10.6276 | 0.4225 | 96.04 | 5 |
| 3 | 125.44 | 0.0784 | 0.3136 | 3.61 | 0.005625 | 289.0 | 3600.0 | 9.9856 | 0.3364 | 96.04 | 6 |
| 4 | 54.76 | 0.4900 | 0.0000 | 3.61 | 0.005776 | 121.0 | 1156.0 | 12.3201 | 0.3136 | 88.36 | 5 |
get_VIF_Table(dff,target)
| Feature | VIF | |
|---|---|---|
| 7 | Parameter 9 | 35.338024 |
| 9 | Parameter 11 | 28.248033 |
| 0 | Parameter 1 | 10.064286 |
| 2 | Parameter 3 | 4.784843 |
| 8 | Parameter 10 | 4.448403 |
| 1 | Parameter 2 | 4.084400 |
| 5 | Parameter 6 | 2.160246 |
| 6 | Parameter 7 | 1.921041 |
| 4 | Parameter 5 | 1.771888 |
| 3 | Parameter 4 | 1.381402 |
columns_to_drop = [target]
Xf = dff.drop(columns_to_drop,axis=1)
yf = dff[target]
n_class = len(np.unique(yf))
n_class
print(f"Before --- Min:{np.min(yf)} and Max:{np.max(yf)}")
yf = yf-3
print(f"After --- Min:{np.min(yf)} and Max:{np.max(yf)}")
Xf_train, Xf_test, yf_train, yf_test = train_test_split(Xf, yf, test_size=0.2, random_state=0)
Xf_train.shape, Xf_test.shape
Before --- Min:3 and Max:8 After --- Min:0 and Max:5
((1279, 10), (320, 10))
dnn_1 = DNN_Model('classification')
m_1 = dnn_1.create_model(Xf_train.shape[1],n_class,**kwargs)
callbacks = LivePlot(2,train_loss='loss',train_metric='accuracy')
# callbacks=[]
dnn_1.train_model(m_1,Xf_train,yf_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(Xf_test,yf_test)
display(Markdown(f"### Test Loss : {result[0]} Test Accuracy : {result[1]}"))
score_board = score_board.append({'Baseline':'Feature engineering only',
'Loss':result[0],
'Metric':result[1],
'Notes':'Loss: Sparse categorical entropy -- Metric : Accuracy '},ignore_index=True)
score_board
| Baseline | Loss | Metric | Notes | |
|---|---|---|---|---|
| 0 | No feature engineering/selection | 0.958858 | 0.621875 | Loss: Sparse categorical entropy -- Metric : Accuracy |
| 1 | Feature engineering only | 0.983865 | 0.621875 | Loss: Sparse categorical entropy -- Metric : Accuracy |
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.feature_selection import SelectFromModel
dfe = df.copy()
x_columns = dfe.drop([target,'Parameter 8'],axis=1).columns
Xe, ye = dfe[x_columns], dfe[target]
print(f"Shape of original data : {Xe.shape}")
clf = ExtraTreesClassifier(n_estimators=50)
clf = clf.fit(Xe, ye)
model = SelectFromModel(clf, prefit=True)
Xe= model.transform(Xe)
print(f"Shape of transformed data after feature selection : {Xe.shape}")
feat_df = pd.DataFrame({'Features':x_columns, 'Importance':clf.feature_importances_}).sort_values(by='Importance',ascending=False)
feat_df
Shape of original data : (1599, 10) Shape of transformed data after feature selection : (1599, 4)
| Features | Importance | |
|---|---|---|
| 9 | Parameter 11 | 0.170855 |
| 8 | Parameter 10 | 0.107795 |
| 6 | Parameter 7 | 0.107598 |
| 1 | Parameter 2 | 0.104754 |
| 2 | Parameter 3 | 0.085657 |
| 3 | Parameter 4 | 0.085429 |
| 0 | Parameter 1 | 0.085281 |
| 5 | Parameter 6 | 0.085088 |
| 4 | Parameter 5 | 0.085012 |
| 7 | Parameter 9 | 0.082532 |
features_chosen = feat_df['Features'][:Xe.shape[1]].values
features_chosen
array(['Parameter 11', 'Parameter 10', 'Parameter 7', 'Parameter 2'],
dtype=object)
Parameter 11, 7 and 10. This has siginificantly reduced the feature space.Xe = dfe[features_chosen]
ye = dfe[target]
n_class = len(np.unique(ye))
n_class
print(f"Before --- Min:{np.min(ye)} and Max:{np.max(ye)}")
ye = ye-3
print(f"After --- Min:{np.min(ye)} and Max:{np.max(ye)}")
Xe_train, Xe_test, ye_train, ye_test = train_test_split(Xe, ye, test_size=0.2, random_state=0)
Xe_train.shape, Xe_test.shape
Before --- Min:3 and Max:8 After --- Min:0 and Max:5
((1279, 4), (320, 4))
scaled = StandardScaler().fit(Xe_train)
Xe_train = scaled.transform(Xe_train)
Xe_test = scaled.transform(Xe_test)
dnn_1 = DNN_Model('classification')
m_1 = dnn_1.create_model(Xe_train.shape[1],n_class,**kwargs)
callbacks = LivePlot(2,train_loss='loss',train_metric='accuracy')
# callbacks=[]
dnn_1.train_model(m_1,Xe_train,ye_train,[loss],[metric],callbacks=[callbacks])
result = dnn_1.test_model(Xe_test,ye_test)
display(Markdown(f"### Test Loss : {result[0]} Test Accuracy : {result[1]}"))
score_board = score_board.append({'Baseline':'Feature selection only',
'Loss':result[0],
'Metric':result[1],
'Notes':'Loss: Sparse categorical entropy -- Metric : Accuracy '},ignore_index=True)
score_board
| Baseline | Loss | Metric | Notes | |
|---|---|---|---|---|
| 0 | No feature engineering/selection | 0.958858 | 0.621875 | Loss: Sparse categorical entropy -- Metric : Accuracy |
| 1 | Feature engineering only | 0.983865 | 0.621875 | Loss: Sparse categorical entropy -- Metric : Accuracy |
| 2 | Feature selection only | 0.956479 | 0.625000 | Loss: Sparse categorical entropy -- Metric : Accuracy |
display(Markdown("#### Score Board from model baselining"))
score_board
| Baseline | Loss | Metric | Notes | |
|---|---|---|---|---|
| 0 | No feature engineering/selection | 0.958858 | 0.621875 | Loss: Sparse categorical entropy -- Metric : Accuracy |
| 1 | Feature engineering only | 0.983865 | 0.621875 | Loss: Sparse categorical entropy -- Metric : Accuracy |
| 2 | Feature selection only | 0.956479 | 0.625000 | Loss: Sparse categorical entropy -- Metric : Accuracy |
Based on the score board above, accuracy is the though marginal is highest for the data with feature selection and hence we will use this as a baseline for our hyper parameter tuning and model training
Activation, BatchNormalization and Dropout.At each stage the parameter determined in the previous stage flows into the next one to override defaults
X_train,X_test,y_train,y_test = Xe_train, Xe_test, ye_train, ye_test
X_train.shape, X_test.shape, n_class
((1279, 4), (320, 4), 6)
hp_score_board = pd.DataFrame(columns=['Phase','Best Value','Hyperparameters'])
def objective_arch(trial):
# A = Activation
# B = BatchNormalization
# D = Dropout
#['A B D'] = ['Activation','BarchNormalization','Dropout']
global loss,metric,X_train,X_test,y_train,y_test
epochs = 50
# loss ='sparse_categorical_crossentropy'
# metric ='accuracy'
lr = 0.001
num_layers = trial.suggest_categorical('num_layers',[1,2,3,4,5,6,7])
arch = trial.suggest_categorical('arch',[' '.join(i) for i in permutations(['A','B','D'],3)])
neurons_per_layer = [trial.suggest_categorical(f'neuron_l{layers}',[10,32,64,128,256,1000]) for layers in range(1,num_layers+1)]
# opts = trial.suggest_categorical('opts',['sgd','adadelta','adam','rmsprop'])
opts = 'adam'
if opts == 'sgd':
optimizer = tf.keras.optimizers.SGD(learning_rate=lr)
elif opts == 'adadelta':
optimizer = tf.keras.optimizers.Adadelta(learning_rate=lr)
elif opts == 'adam':
optimizer = tf.keras.optimizers.Adam(learning_rate=lr)
elif opts == 'rmsprop':
optimizer = tf.keras.optimizers.RMSprop(learning_rate=lr)
kwargs = {'num_layers': num_layers,
'arch' :arch,
'neurons_per_layer':neurons_per_layer,
'batch_size': 10,
'drop_rate_per_layer': [0.2]*num_layers,
'activation': 'relu',
'n_epochs': epochs,
'kernel_init': 'uniform',
'lr': lr,
'opts': optimizer}
dnn_1 = DNN_Model('classification')
m_1 = dnn_1.create_model(X_train.shape[1],n_class)
# callbacks = LivePlot(2,train_loss='mean_squared_error',train_metric='mean_squared_error')
callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,['sparse_categorical_crossentropy'],['accuracy'],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
return result[1]
study_arch = optuna.create_study(direction='maximize',sampler=optuna.samplers.TPESampler())
study_arch.optimize(objective_arch,n_trials=800,n_jobs=-13)
[I 2021-03-10 16:36:47,285] A new study created in memory with name: no-name-326ae07e-274b-424c-bcf8-c4a537863abb [I 2021-03-10 16:37:00,766] Trial 0 finished with value: 0.6343749761581421 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 0 with value: 0.6343749761581421. [I 2021-03-10 16:37:00,793] Trial 3 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 10}. Best is trial 0 with value: 0.6343749761581421. [I 2021-03-10 16:37:00,816] Trial 2 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 10, 'neuron_l3': 10}. Best is trial 0 with value: 0.6343749761581421. [I 2021-03-10 16:37:00,825] Trial 1 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 128, 'neuron_l7': 64}. Best is trial 1 with value: 0.640625. [I 2021-03-10 16:37:13,235] Trial 6 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'D B A', 'neuron_l1': 256}. Best is trial 1 with value: 0.640625. [I 2021-03-10 16:37:13,248] Trial 7 finished with value: 0.6156250238418579 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 128, 'neuron_l5': 32}. Best is trial 1 with value: 0.640625. [I 2021-03-10 16:37:13,262] Trial 4 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 10}. Best is trial 1 with value: 0.640625. [I 2021-03-10 16:37:13,273] Trial 5 finished with value: 0.625 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 1 with value: 0.640625. [I 2021-03-10 16:37:27,406] Trial 8 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 128, 'neuron_l6': 32, 'neuron_l7': 64}. Best is trial 1 with value: 0.640625. [I 2021-03-10 16:37:27,589] Trial 11 finished with value: 0.643750011920929 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 256}. Best is trial 11 with value: 0.643750011920929. [I 2021-03-10 16:37:27,595] Trial 10 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 10}. Best is trial 11 with value: 0.643750011920929. [I 2021-03-10 16:37:28,423] Trial 9 finished with value: 0.653124988079071 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:37:40,854] Trial 12 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 128, 'neuron_l6': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:37:40,992] Trial 13 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:37:41,071] Trial 14 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256, 'neuron_l4': 32, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:37:41,089] Trial 15 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:37:56,281] Trial 16 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:37:56,317] Trial 19 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:37:56,329] Trial 18 finished with value: 0.621874988079071 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:37:56,343] Trial 17 finished with value: 0.6156250238418579 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:09,912] Trial 21 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:09,975] Trial 22 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:09,986] Trial 20 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:09,992] Trial 23 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 10}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:27,121] Trial 27 finished with value: 0.6343749761581421 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:27,377] Trial 25 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:27,394] Trial 26 finished with value: 0.637499988079071 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:27,395] Trial 24 finished with value: 0.6187499761581421 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:41,003] Trial 30 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:41,018] Trial 29 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:41,054] Trial 31 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:38:41,093] Trial 28 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:01,340] Trial 32 finished with value: 0.6499999761581421 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:01,402] Trial 35 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:01,407] Trial 34 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 10}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:01,774] Trial 33 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:15,024] Trial 37 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:15,201] Trial 39 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:15,208] Trial 36 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:15,218] Trial 38 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:29,477] Trial 40 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:30,175] Trial 43 finished with value: 0.621874988079071 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:30,186] Trial 42 finished with value: 0.643750011920929 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:30,251] Trial 41 finished with value: 0.6343749761581421 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:41,211] Trial 44 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:42,408] Trial 47 finished with value: 0.6343749761581421 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:42,519] Trial 45 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:42,615] Trial 46 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:55,272] Trial 48 finished with value: 0.6468750238418579 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:55,838] Trial 49 finished with value: 0.643750011920929 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:55,858] Trial 50 finished with value: 0.6156250238418579 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:39:55,894] Trial 51 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:08,288] Trial 52 finished with value: 0.621874988079071 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:09,169] Trial 53 finished with value: 0.6312500238418579 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:09,349] Trial 55 finished with value: 0.6187499761581421 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:09,466] Trial 54 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:21,544] Trial 56 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:21,608] Trial 57 finished with value: 0.621874988079071 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:21,666] Trial 58 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:21,673] Trial 59 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:35,412] Trial 60 finished with value: 0.6499999761581421 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:36,219] Trial 62 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:36,303] Trial 61 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:36,304] Trial 63 finished with value: 0.6156250238418579 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:47,375] Trial 64 finished with value: 0.621874988079071 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:48,444] Trial 66 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:48,487] Trial 67 finished with value: 0.6499999761581421 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:40:48,993] Trial 65 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:04,845] Trial 68 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:05,890] Trial 71 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:06,020] Trial 70 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:06,048] Trial 69 finished with value: 0.609375 and parameters: {'num_layers': 5, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:20,400] Trial 72 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:21,920] Trial 74 finished with value: 0.6343749761581421 and parameters: {'num_layers': 5, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 256, 'neuron_l5': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:21,950] Trial 73 finished with value: 0.6187499761581421 and parameters: {'num_layers': 5, 'arch': 'B A D', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 256, 'neuron_l5': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:21,958] Trial 75 finished with value: 0.625 and parameters: {'num_layers': 1, 'arch': 'B A D', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:34,139] Trial 76 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:35,831] Trial 78 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:35,866] Trial 77 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:35,892] Trial 79 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:49,741] Trial 80 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:51,282] Trial 83 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:51,284] Trial 82 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:41:51,288] Trial 81 finished with value: 0.643750011920929 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:02,983] Trial 84 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:04,612] Trial 86 finished with value: 0.6343749761581421 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:04,660] Trial 87 finished with value: 0.643750011920929 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:04,779] Trial 85 finished with value: 0.643750011920929 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:18,319] Trial 88 finished with value: 0.6156250238418579 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:20,476] Trial 90 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:20,529] Trial 89 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 10}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:21,443] Trial 91 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:34,777] Trial 92 finished with value: 0.621874988079071 and parameters: {'num_layers': 1, 'arch': 'D B A', 'neuron_l1': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:37,272] Trial 94 finished with value: 0.621874988079071 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:37,327] Trial 93 finished with value: 0.643750011920929 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:38,189] Trial 95 finished with value: 0.6499999761581421 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:53,128] Trial 96 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:54,706] Trial 98 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:54,838] Trial 97 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:42:55,149] Trial 99 finished with value: 0.6187499761581421 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:08,195] Trial 100 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:09,571] Trial 103 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:09,605] Trial 101 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:09,620] Trial 102 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:21,161] Trial 104 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:22,565] Trial 106 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:22,601] Trial 105 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:22,636] Trial 107 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:34,952] Trial 108 finished with value: 0.6312500238418579 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:37,027] Trial 110 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 128, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:37,123] Trial 111 finished with value: 0.6312500238418579 and parameters: {'num_layers': 1, 'arch': 'D B A', 'neuron_l1': 10}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:37,142] Trial 109 finished with value: 0.628125011920929 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:53,162] Trial 112 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 128, 'neuron_l6': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:54,872] Trial 115 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'D B A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:54,953] Trial 113 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:43:56,056] Trial 114 finished with value: 0.628125011920929 and parameters: {'num_layers': 1, 'arch': 'D B A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:10,583] Trial 116 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:12,416] Trial 117 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 64, 'neuron_l7': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:12,748] Trial 118 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 10, 'neuron_l5': 128, 'neuron_l6': 64, 'neuron_l7': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:13,394] Trial 119 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 10, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:29,315] Trial 120 finished with value: 0.606249988079071 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:30,596] Trial 121 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:30,675] Trial 122 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:30,731] Trial 123 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:46,760] Trial 124 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:48,380] Trial 127 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:48,465] Trial 126 finished with value: 0.625 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:44:48,576] Trial 125 finished with value: 0.6156250238418579 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:03,983] Trial 128 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:05,409] Trial 130 finished with value: 0.625 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:05,438] Trial 129 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:06,264] Trial 131 finished with value: 0.621874988079071 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:21,839] Trial 132 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:23,124] Trial 133 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:23,214] Trial 135 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:23,306] Trial 134 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:39,136] Trial 136 finished with value: 0.6156250238418579 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:40,184] Trial 138 finished with value: 0.625 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:40,311] Trial 137 finished with value: 0.628125011920929 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:40,344] Trial 139 finished with value: 0.6156250238418579 and parameters: {'num_layers': 1, 'arch': 'D B A', 'neuron_l1': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:55,738] Trial 140 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:56,864] Trial 141 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:56,866] Trial 143 finished with value: 0.6468750238418579 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:45:56,928] Trial 142 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:11,605] Trial 144 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:13,163] Trial 146 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 128, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:13,358] Trial 145 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 128, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:13,485] Trial 147 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 128, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:28,451] Trial 148 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:28,511] Trial 149 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:29,459] Trial 151 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 128, 'neuron_l2': 10}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:29,507] Trial 150 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:45,224] Trial 153 finished with value: 0.625 and parameters: {'num_layers': 5, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:45,240] Trial 152 finished with value: 0.609375 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:45,723] Trial 155 finished with value: 0.6343749761581421 and parameters: {'num_layers': 5, 'arch': 'A B D', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:46:46,278] Trial 154 finished with value: 0.6343749761581421 and parameters: {'num_layers': 5, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:00,609] Trial 156 finished with value: 0.628125011920929 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 10}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:00,642] Trial 158 finished with value: 0.6312500238418579 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:00,686] Trial 157 finished with value: 0.6187499761581421 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 10}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:00,692] Trial 159 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:17,073] Trial 162 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:17,549] Trial 161 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:17,634] Trial 160 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:17,958] Trial 163 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:32,917] Trial 164 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:34,310] Trial 166 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:34,317] Trial 167 finished with value: 0.643750011920929 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:34,325] Trial 165 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:48,834] Trial 168 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:50,532] Trial 169 finished with value: 0.6156250238418579 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:50,562] Trial 170 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:47:50,660] Trial 171 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:05,189] Trial 172 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:07,281] Trial 175 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:07,293] Trial 174 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:07,318] Trial 173 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:20,079] Trial 176 finished with value: 0.6468750238418579 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:21,384] Trial 177 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:21,581] Trial 178 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:21,809] Trial 179 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:34,731] Trial 180 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:36,294] Trial 181 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:36,322] Trial 183 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:36,334] Trial 182 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:48,060] Trial 184 finished with value: 0.621874988079071 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:50,045] Trial 186 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:50,049] Trial 187 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:48:50,344] Trial 185 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:01,803] Trial 188 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:03,391] Trial 191 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:03,440] Trial 190 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:03,448] Trial 189 finished with value: 0.621874988079071 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:15,353] Trial 192 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:17,043] Trial 193 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:17,055] Trial 195 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:17,081] Trial 194 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:28,029] Trial 196 finished with value: 0.640625 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:30,182] Trial 199 finished with value: 0.6468750238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:30,220] Trial 198 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:30,267] Trial 197 finished with value: 0.6312500238418579 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:40,029] Trial 200 finished with value: 0.621874988079071 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:42,400] Trial 201 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:42,448] Trial 202 finished with value: 0.6187499761581421 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:42,535] Trial 203 finished with value: 0.612500011920929 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:53,799] Trial 204 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:58,059] Trial 207 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:58,196] Trial 206 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:49:58,430] Trial 205 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:07,922] Trial 208 finished with value: 0.6499999761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:11,269] Trial 209 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 256}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:11,525] Trial 210 finished with value: 0.621874988079071 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:11,964] Trial 211 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:21,898] Trial 212 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:26,165] Trial 213 finished with value: 0.653124988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:26,179] Trial 214 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:27,082] Trial 215 finished with value: 0.6156250238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:33,659] Trial 216 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:39,543] Trial 218 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:39,921] Trial 219 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:40,681] Trial 217 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:47,459] Trial 220 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:53,925] Trial 221 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:53,951] Trial 222 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:54,407] Trial 223 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 10}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:50:59,897] Trial 224 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:08,566] Trial 225 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:08,574] Trial 226 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:09,232] Trial 227 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:14,389] Trial 228 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 32}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:23,250] Trial 230 finished with value: 0.609375 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:23,449] Trial 231 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:23,609] Trial 229 finished with value: 0.6000000238418579 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:27,897] Trial 232 finished with value: 0.6187499761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 128}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:39,035] Trial 234 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:39,149] Trial 233 finished with value: 0.6499999761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:39,153] Trial 235 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:43,560] Trial 236 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:54,889] Trial 239 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:54,899] Trial 238 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:54,975] Trial 237 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:51:58,722] Trial 240 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:52:11,193] Trial 241 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 10, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 9 with value: 0.653124988079071. [I 2021-03-10 16:52:11,196] Trial 242 finished with value: 0.659375011920929 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:11,198] Trial 243 finished with value: 0.625 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:13,978] Trial 244 finished with value: 0.6312500238418579 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:26,512] Trial 245 finished with value: 0.6187499761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:26,603] Trial 246 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:26,614] Trial 247 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:28,776] Trial 248 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:42,557] Trial 250 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:42,638] Trial 251 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:43,050] Trial 249 finished with value: 0.621874988079071 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:43,107] Trial 252 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:56,165] Trial 256 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 32, 'neuron_l5': 32, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:56,220] Trial 254 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:56,224] Trial 253 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:52:56,227] Trial 255 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:09,149] Trial 258 finished with value: 0.625 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:09,232] Trial 260 finished with value: 0.621874988079071 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:09,252] Trial 259 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:09,571] Trial 257 finished with value: 0.6156250238418579 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:23,041] Trial 261 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:23,096] Trial 262 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:23,103] Trial 264 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:23,104] Trial 263 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:36,486] Trial 265 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:36,524] Trial 267 finished with value: 0.6187499761581421 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:36,535] Trial 268 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:37,008] Trial 266 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:48,617] Trial 272 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:48,625] Trial 270 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:48,635] Trial 271 finished with value: 0.6187499761581421 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:53:48,637] Trial 269 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:02,138] Trial 273 finished with value: 0.6312500238418579 and parameters: {'num_layers': 5, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 64, 'neuron_l5': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:02,279] Trial 276 finished with value: 0.6312500238418579 and parameters: {'num_layers': 5, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 64, 'neuron_l5': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:02,315] Trial 274 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:02,724] Trial 275 finished with value: 0.621874988079071 and parameters: {'num_layers': 5, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 64, 'neuron_l5': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:13,866] Trial 277 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:13,930] Trial 279 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:13,979] Trial 278 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:13,995] Trial 280 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:29,362] Trial 283 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:29,456] Trial 284 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:29,486] Trial 282 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:29,507] Trial 281 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:41,631] Trial 288 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:41,647] Trial 286 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:41,663] Trial 285 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:41,677] Trial 287 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:55,071] Trial 290 finished with value: 0.640625 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:55,391] Trial 291 finished with value: 0.625 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:55,456] Trial 292 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:54:55,991] Trial 289 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:07,620] Trial 293 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:08,546] Trial 295 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:08,599] Trial 294 finished with value: 0.6468750238418579 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:08,608] Trial 296 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 128, 'neuron_l3': 10, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:21,748] Trial 297 finished with value: 0.621874988079071 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:22,328] Trial 298 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:22,359] Trial 300 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:22,425] Trial 299 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:34,917] Trial 301 finished with value: 0.6468750238418579 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:36,142] Trial 304 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:36,208] Trial 302 finished with value: 0.640625 and parameters: {'num_layers': 1, 'arch': 'A B D', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:36,211] Trial 303 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'A B D', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:49,272] Trial 305 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 128, 'neuron_l2': 1000, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:50,151] Trial 307 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:50,187] Trial 306 finished with value: 0.621874988079071 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:55:50,616] Trial 308 finished with value: 0.6468750238418579 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:02,953] Trial 309 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:04,277] Trial 312 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:04,326] Trial 310 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:04,411] Trial 311 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:15,532] Trial 313 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:17,155] Trial 316 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:17,157] Trial 315 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:17,196] Trial 314 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:29,998] Trial 317 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:31,275] Trial 318 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:31,420] Trial 320 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:31,443] Trial 319 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:43,191] Trial 321 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:44,680] Trial 323 finished with value: 0.6499999761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 64, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:44,846] Trial 324 finished with value: 0.6499999761581421 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:44,943] Trial 322 finished with value: 0.621874988079071 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:58,451] Trial 325 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:59,484] Trial 327 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:59,536] Trial 326 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:56:59,574] Trial 328 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:10,498] Trial 329 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 64, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:11,996] Trial 330 finished with value: 0.6156250238418579 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:12,016] Trial 331 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:12,208] Trial 332 finished with value: 0.6187499761581421 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:24,613] Trial 333 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:26,706] Trial 334 finished with value: 0.6156250238418579 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:26,732] Trial 335 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 64, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:26,740] Trial 336 finished with value: 0.6187499761581421 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:39,876] Trial 337 finished with value: 0.612500011920929 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 64, 'neuron_l6': 64, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:41,523] Trial 338 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:41,604] Trial 340 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 64, 'neuron_l6': 64, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:41,881] Trial 339 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:54,635] Trial 341 finished with value: 0.659375011920929 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:56,484] Trial 343 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:56,577] Trial 342 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'D B A', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:57:56,585] Trial 344 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:08,810] Trial 345 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 1000, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:10,431] Trial 346 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:10,469] Trial 348 finished with value: 0.6156250238418579 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:10,525] Trial 347 finished with value: 0.621874988079071 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:24,632] Trial 349 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:26,042] Trial 351 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:26,087] Trial 352 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:26,132] Trial 350 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 32, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:39,915] Trial 353 finished with value: 0.6187499761581421 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:40,723] Trial 355 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:41,234] Trial 356 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:41,240] Trial 354 finished with value: 0.612500011920929 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:54,537] Trial 357 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:55,170] Trial 360 finished with value: 0.6468750238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:55,302] Trial 359 finished with value: 0.643750011920929 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:58:55,319] Trial 358 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:10,554] Trial 361 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:11,700] Trial 364 finished with value: 0.643750011920929 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:11,731] Trial 362 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:12,315] Trial 363 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:24,171] Trial 365 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:25,377] Trial 367 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:25,596] Trial 366 finished with value: 0.6499999761581421 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:25,647] Trial 368 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:39,210] Trial 369 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:40,428] Trial 370 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:40,504] Trial 371 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:40,508] Trial 372 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:52,858] Trial 373 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:54,496] Trial 376 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:54,599] Trial 374 finished with value: 0.643750011920929 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 16:59:54,630] Trial 375 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:06,740] Trial 377 finished with value: 0.609375 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:08,588] Trial 380 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:08,720] Trial 379 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:08,890] Trial 378 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:21,450] Trial 381 finished with value: 0.621874988079071 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:24,391] Trial 382 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 32, 'neuron_l3': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:24,454] Trial 383 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 128, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:24,471] Trial 384 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:34,450] Trial 385 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:37,772] Trial 387 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 32, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:37,830] Trial 388 finished with value: 0.6156250238418579 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:38,178] Trial 386 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:47,417] Trial 389 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:52,719] Trial 390 finished with value: 0.6187499761581421 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:52,720] Trial 392 finished with value: 0.640625 and parameters: {'num_layers': 5, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:52,769] Trial 391 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:00:59,477] Trial 393 finished with value: 0.6156250238418579 and parameters: {'num_layers': 1, 'arch': 'B A D', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:06,168] Trial 395 finished with value: 0.6312500238418579 and parameters: {'num_layers': 1, 'arch': 'A B D', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:06,392] Trial 394 finished with value: 0.6343749761581421 and parameters: {'num_layers': 1, 'arch': 'A B D', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:06,717] Trial 396 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:13,660] Trial 397 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'A D B', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:20,742] Trial 398 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:21,018] Trial 399 finished with value: 0.6187499761581421 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:21,087] Trial 400 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:25,998] Trial 401 finished with value: 0.6468750238418579 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:35,337] Trial 403 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 32, 'neuron_l3': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:35,344] Trial 402 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:36,259] Trial 404 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:42,724] Trial 405 finished with value: 0.621874988079071 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:52,124] Trial 406 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:52,484] Trial 407 finished with value: 0.612500011920929 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:53,486] Trial 408 finished with value: 0.6499999761581421 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:01:57,548] Trial 409 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 32, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:07,661] Trial 410 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:07,737] Trial 411 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:09,803] Trial 412 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 32, 'neuron_l6': 64, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:12,633] Trial 413 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 64, 'neuron_l6': 64, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:21,568] Trial 414 finished with value: 0.637499988079071 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:21,674] Trial 415 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:22,799] Trial 416 finished with value: 0.6156250238418579 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:24,734] Trial 417 finished with value: 0.6156250238418579 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:37,636] Trial 418 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:38,253] Trial 419 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:38,541] Trial 420 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:40,507] Trial 421 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:51,963] Trial 422 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:53,286] Trial 424 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:53,296] Trial 423 finished with value: 0.621874988079071 and parameters: {'num_layers': 7, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 256, 'neuron_l7': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:02:54,076] Trial 425 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:07,589] Trial 426 finished with value: 0.621874988079071 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 128, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 256, 'neuron_l6': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:09,477] Trial 427 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 128, 'neuron_l5': 64, 'neuron_l6': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:09,567] Trial 429 finished with value: 0.6187499761581421 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:09,879] Trial 428 finished with value: 0.6187499761581421 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:22,997] Trial 430 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:24,842] Trial 431 finished with value: 0.643750011920929 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 10, 'neuron_l2': 32, 'neuron_l3': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:24,866] Trial 433 finished with value: 0.6031249761581421 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 10, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:24,885] Trial 432 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'B D A', 'neuron_l1': 10, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:37,949] Trial 434 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:39,644] Trial 437 finished with value: 0.6468750238418579 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:39,725] Trial 435 finished with value: 0.612500011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:39,904] Trial 436 finished with value: 0.621874988079071 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:51,787] Trial 438 finished with value: 0.6156250238418579 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:53,575] Trial 439 finished with value: 0.6187499761581421 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:53,588] Trial 441 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 128, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:03:53,599] Trial 440 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 128, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:07,435] Trial 442 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:08,633] Trial 445 finished with value: 0.6187499761581421 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:08,680] Trial 444 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:08,699] Trial 443 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:23,941] Trial 446 finished with value: 0.643750011920929 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:25,978] Trial 447 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:25,993] Trial 449 finished with value: 0.6499999761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:25,994] Trial 448 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:40,572] Trial 450 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:41,858] Trial 451 finished with value: 0.606249988079071 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:41,861] Trial 453 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:41,893] Trial 452 finished with value: 0.6156250238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:55,446] Trial 454 finished with value: 0.6499999761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:58,205] Trial 457 finished with value: 0.609375 and parameters: {'num_layers': 5, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:58,338] Trial 456 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:04:58,406] Trial 455 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:12,183] Trial 458 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:16,107] Trial 461 finished with value: 0.637499988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:16,125] Trial 459 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:17,054] Trial 460 finished with value: 0.6499999761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:30,197] Trial 462 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:34,077] Trial 463 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:34,160] Trial 464 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:34,165] Trial 465 finished with value: 0.6499999761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:44,539] Trial 466 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:49,980] Trial 467 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:49,983] Trial 469 finished with value: 0.643750011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:49,998] Trial 468 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:05:59,255] Trial 470 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:04,657] Trial 471 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:04,666] Trial 472 finished with value: 0.609375 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:04,667] Trial 473 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:13,567] Trial 474 finished with value: 0.621874988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:20,990] Trial 475 finished with value: 0.621874988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:21,080] Trial 476 finished with value: 0.6468750238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:21,093] Trial 477 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 256, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:29,238] Trial 478 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:36,709] Trial 480 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:36,756] Trial 481 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:36,759] Trial 479 finished with value: 0.637499988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 64, 'neuron_l7': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:44,420] Trial 482 finished with value: 0.6468750238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:52,842] Trial 483 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:52,960] Trial 484 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:53,552] Trial 485 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:06:58,073] Trial 486 finished with value: 0.65625 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:07,554] Trial 488 finished with value: 0.609375 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:07,580] Trial 489 finished with value: 0.621874988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:07,617] Trial 487 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:12,422] Trial 490 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:23,553] Trial 491 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:23,598] Trial 493 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 10, 'neuron_l4': 128, 'neuron_l5': 64, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:24,132] Trial 492 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 10, 'neuron_l4': 128, 'neuron_l5': 64, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:26,659] Trial 494 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:41,017] Trial 497 finished with value: 0.6187499761581421 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:41,026] Trial 496 finished with value: 0.621874988079071 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:41,027] Trial 495 finished with value: 0.637499988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:43,705] Trial 498 finished with value: 0.628125011920929 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:54,889] Trial 499 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:55,080] Trial 501 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:55,693] Trial 500 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:07:56,710] Trial 502 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:08,332] Trial 503 finished with value: 0.6156250238418579 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:09,250] Trial 504 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:09,311] Trial 505 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:10,166] Trial 506 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 128, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:21,734] Trial 507 finished with value: 0.621874988079071 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:22,817] Trial 509 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 128, 'neuron_l6': 64, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:22,985] Trial 508 finished with value: 0.621874988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 64, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:23,426] Trial 510 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:34,375] Trial 511 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:35,299] Trial 513 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:35,359] Trial 514 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:35,377] Trial 512 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:48,147] Trial 515 finished with value: 0.65625 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:49,609] Trial 517 finished with value: 0.628125011920929 and parameters: {'num_layers': 5, 'arch': 'B A D', 'neuron_l1': 128, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:49,692] Trial 516 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:08:50,006] Trial 518 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 10, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:01,498] Trial 519 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 256, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:03,063] Trial 522 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:03,072] Trial 520 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:03,113] Trial 521 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:15,259] Trial 523 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:17,297] Trial 525 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:17,400] Trial 524 finished with value: 0.612500011920929 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:18,007] Trial 526 finished with value: 0.637499988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:29,494] Trial 527 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:31,783] Trial 528 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:31,828] Trial 529 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 32, 'neuron_l5': 64, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:31,918] Trial 530 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:42,263] Trial 531 finished with value: 0.643750011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 32, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:44,070] Trial 532 finished with value: 0.621874988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:44,174] Trial 534 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:44,620] Trial 533 finished with value: 0.6187499761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:57,248] Trial 535 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:59,033] Trial 536 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:59,061] Trial 538 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:09:59,064] Trial 537 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 128, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:10,210] Trial 539 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 128, 'neuron_l7': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:12,216] Trial 541 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:12,387] Trial 542 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:12,706] Trial 540 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 256, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:25,512] Trial 543 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:26,736] Trial 545 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:26,766] Trial 546 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:26,826] Trial 544 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:38,276] Trial 547 finished with value: 0.609375 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 32, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:40,360] Trial 549 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:40,403] Trial 548 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:40,944] Trial 550 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:52,760] Trial 551 finished with value: 0.6187499761581421 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:53,792] Trial 552 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:53,817] Trial 553 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:10:53,831] Trial 554 finished with value: 0.6187499761581421 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:05,465] Trial 555 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:07,504] Trial 557 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:07,539] Trial 556 finished with value: 0.621874988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:07,772] Trial 558 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:18,880] Trial 559 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:20,682] Trial 562 finished with value: 0.628125011920929 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:20,718] Trial 561 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:20,736] Trial 560 finished with value: 0.653124988079071 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:34,160] Trial 563 finished with value: 0.612500011920929 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 64, 'neuron_l5': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:35,916] Trial 564 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:35,934] Trial 565 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 256, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:36,261] Trial 566 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:49,065] Trial 567 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:50,460] Trial 569 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:50,773] Trial 568 finished with value: 0.6187499761581421 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:11:50,800] Trial 570 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:04,803] Trial 571 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:06,539] Trial 572 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 64, 'neuron_l5': 64, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:06,633] Trial 574 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:07,565] Trial 573 finished with value: 0.6156250238418579 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 64, 'neuron_l6': 128, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:22,320] Trial 575 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 128, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:23,507] Trial 577 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:23,765] Trial 576 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:24,379] Trial 578 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:38,996] Trial 579 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:40,929] Trial 582 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 32, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:41,000] Trial 580 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:41,659] Trial 581 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:55,804] Trial 583 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:57,156] Trial 585 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:57,162] Trial 584 finished with value: 0.637499988079071 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:12:57,797] Trial 586 finished with value: 0.6312500238418579 and parameters: {'num_layers': 1, 'arch': 'B D A', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:08,748] Trial 587 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:10,949] Trial 588 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 128, 'neuron_l6': 256, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:10,953] Trial 590 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 128, 'neuron_l6': 256, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:11,811] Trial 589 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 128, 'neuron_l6': 256, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:24,999] Trial 591 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 128, 'neuron_l6': 256, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:27,097] Trial 593 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:27,353] Trial 592 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:27,437] Trial 594 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:37,343] Trial 595 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:41,518] Trial 596 finished with value: 0.643750011920929 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:41,963] Trial 597 finished with value: 0.643750011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:42,624] Trial 598 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:56,584] Trial 599 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:58,964] Trial 600 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:59,093] Trial 601 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:13:59,589] Trial 602 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 10, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:08,199] Trial 603 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'B A D', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:12,004] Trial 606 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 32, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:12,071] Trial 605 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 10, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:12,710] Trial 604 finished with value: 0.6187499761581421 and parameters: {'num_layers': 1, 'arch': 'D A B', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:22,518] Trial 607 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 32, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:25,135] Trial 609 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:25,146] Trial 608 finished with value: 0.637499988079071 and parameters: {'num_layers': 1, 'arch': 'D B A', 'neuron_l1': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:25,729] Trial 610 finished with value: 0.6187499761581421 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:34,985] Trial 611 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:40,716] Trial 613 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:40,785] Trial 614 finished with value: 0.6000000238418579 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:41,240] Trial 612 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:51,920] Trial 615 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:55,298] Trial 617 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:55,409] Trial 616 finished with value: 0.625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 128, 'neuron_l5': 32, 'neuron_l6': 128, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:14:55,412] Trial 618 finished with value: 0.621874988079071 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:04,179] Trial 619 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 64, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:11,005] Trial 620 finished with value: 0.640625 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:11,453] Trial 622 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:11,790] Trial 621 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'A B D', 'neuron_l1': 32, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:17,442] Trial 623 finished with value: 0.643750011920929 and parameters: {'num_layers': 1, 'arch': 'A B D', 'neuron_l1': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:22,692] Trial 624 finished with value: 0.6187499761581421 and parameters: {'num_layers': 1, 'arch': 'A B D', 'neuron_l1': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:23,000] Trial 625 finished with value: 0.6312500238418579 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:23,045] Trial 626 finished with value: 0.6187499761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:28,854] Trial 627 finished with value: 0.637499988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:37,347] Trial 628 finished with value: 0.612500011920929 and parameters: {'num_layers': 2, 'arch': 'A B D', 'neuron_l1': 64, 'neuron_l2': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:37,906] Trial 630 finished with value: 0.6156250238418579 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:38,093] Trial 629 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 32, 'neuron_l5': 128, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:42,757] Trial 631 finished with value: 0.6468750238418579 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:50,955] Trial 632 finished with value: 0.6468750238418579 and parameters: {'num_layers': 7, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:51,338] Trial 633 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:51,343] Trial 634 finished with value: 0.6343749761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:15:55,343] Trial 635 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:03,659] Trial 638 finished with value: 0.637499988079071 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:03,786] Trial 637 finished with value: 0.643750011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:04,156] Trial 636 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:06,771] Trial 639 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 10, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:19,272] Trial 640 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:19,277] Trial 642 finished with value: 0.628125011920929 and parameters: {'num_layers': 3, 'arch': 'B D A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:19,365] Trial 641 finished with value: 0.6499999761581421 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:22,187] Trial 643 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 128, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:33,614] Trial 645 finished with value: 0.625 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:33,679] Trial 644 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:34,083] Trial 646 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 32, 'neuron_l3': 10, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:35,063] Trial 647 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:45,041] Trial 650 finished with value: 0.65625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:45,043] Trial 648 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 10, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:45,071] Trial 649 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:16:45,881] Trial 651 finished with value: 0.6156250238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 1000, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:02,956] Trial 653 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:03,339] Trial 652 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:03,777] Trial 654 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:03,778] Trial 655 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:15,319] Trial 656 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:15,616] Trial 657 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:15,758] Trial 658 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 1000, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:15,808] Trial 659 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 1000, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:32,210] Trial 660 finished with value: 0.653124988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:32,570] Trial 662 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:32,837] Trial 663 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:33,294] Trial 661 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:45,227] Trial 664 finished with value: 0.6499999761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:45,280] Trial 665 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:45,282] Trial 666 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:17:46,051] Trial 667 finished with value: 0.606249988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:01,358] Trial 670 finished with value: 0.6156250238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:01,732] Trial 668 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:02,145] Trial 669 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:02,962] Trial 671 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:15,443] Trial 672 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:16,188] Trial 673 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:16,360] Trial 674 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:16,617] Trial 675 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:30,145] Trial 676 finished with value: 0.6187499761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:32,935] Trial 678 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:32,987] Trial 677 finished with value: 0.6468750238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:33,561] Trial 679 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:47,191] Trial 680 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:48,799] Trial 681 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:48,953] Trial 683 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:18:49,086] Trial 682 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:03,609] Trial 684 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:06,129] Trial 685 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:06,554] Trial 687 finished with value: 0.6156250238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:07,016] Trial 686 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:21,243] Trial 688 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:23,258] Trial 689 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:23,597] Trial 691 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:23,610] Trial 690 finished with value: 0.653124988079071 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:38,044] Trial 692 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:39,369] Trial 693 finished with value: 0.6499999761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:40,892] Trial 694 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:40,920] Trial 695 finished with value: 0.621874988079071 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:57,121] Trial 696 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:57,388] Trial 697 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:58,339] Trial 698 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:19:58,417] Trial 699 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:12,250] Trial 700 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:12,253] Trial 701 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:13,553] Trial 702 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:13,589] Trial 703 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:29,544] Trial 705 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:30,375] Trial 706 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:30,401] Trial 704 finished with value: 0.643750011920929 and parameters: {'num_layers': 6, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:30,476] Trial 707 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:41,197] Trial 708 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 64, 'neuron_l6': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:42,358] Trial 709 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:42,629] Trial 710 finished with value: 0.6156250238418579 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 64, 'neuron_l5': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:42,684] Trial 711 finished with value: 0.609375 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:20:59,749] Trial 712 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:00,348] Trial 713 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:00,967] Trial 714 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:01,302] Trial 715 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:11,031] Trial 716 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 10, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:11,184] Trial 717 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:12,514] Trial 718 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 10, 'neuron_l6': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:13,062] Trial 719 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 64, 'neuron_l3': 32, 'neuron_l4': 256, 'neuron_l5': 10, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:29,251] Trial 720 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:29,283] Trial 721 finished with value: 0.640625 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 10, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:30,445] Trial 722 finished with value: 0.640625 and parameters: {'num_layers': 5, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:30,512] Trial 723 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:41,287] Trial 724 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:41,311] Trial 725 finished with value: 0.621874988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:42,079] Trial 726 finished with value: 0.643750011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:42,139] Trial 727 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:58,889] Trial 729 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:59,077] Trial 728 finished with value: 0.6187499761581421 and parameters: {'num_layers': 4, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:21:59,615] Trial 731 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'B A D', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 32, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:00,393] Trial 730 finished with value: 0.6343749761581421 and parameters: {'num_layers': 3, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 256, 'neuron_l3': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:13,522] Trial 734 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:13,579] Trial 733 finished with value: 0.6499999761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:13,599] Trial 732 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:14,608] Trial 735 finished with value: 0.628125011920929 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:31,542] Trial 736 finished with value: 0.621874988079071 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:31,550] Trial 738 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:31,608] Trial 739 finished with value: 0.6312500238418579 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:31,611] Trial 737 finished with value: 0.637499988079071 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 1000, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:46,098] Trial 743 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:46,671] Trial 742 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:46,708] Trial 740 finished with value: 0.6468750238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:22:47,532] Trial 741 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:03,435] Trial 746 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:03,818] Trial 744 finished with value: 0.640625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:03,833] Trial 745 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:04,682] Trial 747 finished with value: 0.625 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:17,670] Trial 748 finished with value: 0.6156250238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:17,693] Trial 749 finished with value: 0.640625 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 128, 'neuron_l5': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:17,718] Trial 750 finished with value: 0.640625 and parameters: {'num_layers': 5, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 128, 'neuron_l3': 64, 'neuron_l4': 128, 'neuron_l5': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:18,379] Trial 751 finished with value: 0.6312500238418579 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:34,470] Trial 752 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:34,781] Trial 755 finished with value: 0.6156250238418579 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:34,841] Trial 754 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:34,865] Trial 753 finished with value: 0.643750011920929 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 256, 'neuron_l4': 64, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:48,358] Trial 758 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 10, 'neuron_l5': 32, 'neuron_l6': 64, 'neuron_l7': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:48,377] Trial 757 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 1000, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:48,436] Trial 759 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:23:48,438] Trial 756 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 128, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:03,840] Trial 760 finished with value: 0.6343749761581421 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:03,880] Trial 763 finished with value: 0.6156250238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:03,908] Trial 761 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:04,024] Trial 762 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 10, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:18,780] Trial 765 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:18,840] Trial 764 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 10, 'neuron_l2': 10, 'neuron_l3': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:18,874] Trial 766 finished with value: 0.609375 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 32, 'neuron_l4': 1000, 'neuron_l5': 256, 'neuron_l6': 256}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:19,606] Trial 767 finished with value: 0.628125011920929 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 128, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:33,580] Trial 769 finished with value: 0.6156250238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:33,630] Trial 770 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:33,719] Trial 768 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 256, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:35,087] Trial 771 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:49,296] Trial 775 finished with value: 0.6312500238418579 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:49,375] Trial 773 finished with value: 0.621874988079071 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:49,407] Trial 774 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 1000, 'neuron_l5': 128, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:24:50,119] Trial 772 finished with value: 0.640625 and parameters: {'num_layers': 3, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:04,320] Trial 776 finished with value: 0.6499999761581421 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:04,641] Trial 778 finished with value: 0.6156250238418579 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 1000, 'neuron_l2': 64, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 32, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:04,704] Trial 779 finished with value: 0.612500011920929 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 10, 'neuron_l3': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:04,717] Trial 777 finished with value: 0.6187499761581421 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:20,892] Trial 780 finished with value: 0.637499988079071 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:21,916] Trial 783 finished with value: 0.637499988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:21,931] Trial 781 finished with value: 0.625 and parameters: {'num_layers': 3, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 64, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:22,835] Trial 782 finished with value: 0.606249988079071 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:37,419] Trial 784 finished with value: 0.609375 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:38,698] Trial 787 finished with value: 0.6187499761581421 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:39,027] Trial 786 finished with value: 0.6468750238418579 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:39,095] Trial 785 finished with value: 0.6156250238418579 and parameters: {'num_layers': 3, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 10, 'neuron_l3': 64}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:55,622] Trial 788 finished with value: 0.640625 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 64, 'neuron_l2': 1000, 'neuron_l3': 128, 'neuron_l4': 64, 'neuron_l5': 256, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:56,361] Trial 789 finished with value: 0.628125011920929 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 1000, 'neuron_l2': 1000, 'neuron_l3': 256, 'neuron_l4': 1000, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:56,463] Trial 790 finished with value: 0.653124988079071 and parameters: {'num_layers': 2, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:25:56,546] Trial 791 finished with value: 0.6499999761581421 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:26:11,527] Trial 792 finished with value: 0.6156250238418579 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:26:13,453] Trial 793 finished with value: 0.628125011920929 and parameters: {'num_layers': 2, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:26:13,491] Trial 794 finished with value: 0.6312500238418579 and parameters: {'num_layers': 4, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:26:13,496] Trial 795 finished with value: 0.6187499761581421 and parameters: {'num_layers': 7, 'arch': 'D A B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 256, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 32}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:26:29,307] Trial 796 finished with value: 0.637499988079071 and parameters: {'num_layers': 7, 'arch': 'B A D', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 256, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000, 'neuron_l7': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:26:30,507] Trial 797 finished with value: 0.6343749761581421 and parameters: {'num_layers': 4, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 10}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:26:30,629] Trial 798 finished with value: 0.625 and parameters: {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929. [I 2021-03-10 17:26:30,635] Trial 799 finished with value: 0.6343749761581421 and parameters: {'num_layers': 6, 'arch': 'D B A', 'neuron_l1': 256, 'neuron_l2': 128, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000}. Best is trial 242 with value: 0.659375011920929.
arch_best_params = study_arch.best_params
display(Markdown("#### Least recorded Loss :"))
print(study_arch.best_value)
display(Markdown("#### Best set of parameters to determine the architecture :"))
print(arch_best_params)
hp_score_board = hp_score_board.append({'Phase':'Architecture Selection',
'Best Value':study_arch.best_value,
'Hyperparameters':f'{study_arch.best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.659375 | {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000} |
optuna.visualization.plot_parallel_coordinate(study_arch)
optuna.visualization.plot_optimization_history(study_arch)
study_arch.trials_dataframe()
| number | value | datetime_start | datetime_complete | duration | params_arch | params_neuron_l1 | params_neuron_l2 | params_neuron_l3 | params_neuron_l4 | params_neuron_l5 | params_neuron_l6 | params_neuron_l7 | params_num_layers | state | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.634375 | 2021-03-10 16:36:47.289501 | 2021-03-10 16:37:00.766092 | 0 days 00:00:13.476591 | D A B | 256 | NaN | NaN | NaN | NaN | NaN | NaN | 1 | COMPLETE |
| 1 | 1 | 0.640625 | 2021-03-10 16:36:47.291009 | 2021-03-10 16:37:00.824929 | 0 days 00:00:13.533920 | D A B | 10 | 128.0 | 1000.0 | 128.0 | 256.0 | 128.0 | 64.0 | 7 | COMPLETE |
| 2 | 2 | 0.634375 | 2021-03-10 16:36:47.292853 | 2021-03-10 16:37:00.815707 | 0 days 00:00:13.522854 | B A D | 32 | 10.0 | 10.0 | NaN | NaN | NaN | NaN | 3 | COMPLETE |
| 3 | 3 | 0.621875 | 2021-03-10 16:36:47.293888 | 2021-03-10 16:37:00.792635 | 0 days 00:00:13.498747 | D B A | 32 | 128.0 | 10.0 | NaN | NaN | NaN | NaN | 3 | COMPLETE |
| 4 | 4 | 0.637500 | 2021-03-10 16:37:00.771667 | 2021-03-10 16:37:13.261870 | 0 days 00:00:12.490203 | B D A | 10 | NaN | NaN | NaN | NaN | NaN | NaN | 1 | COMPLETE |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 795 | 795 | 0.618750 | 2021-03-10 17:25:56.554992 | 2021-03-10 17:26:13.496020 | 0 days 00:00:16.941028 | D A B | 256 | 128.0 | 1000.0 | 256.0 | 1000.0 | 1000.0 | 32.0 | 7 | COMPLETE |
| 796 | 796 | 0.637500 | 2021-03-10 17:26:11.537842 | 2021-03-10 17:26:29.307276 | 0 days 00:00:17.769434 | B A D | 256 | 128.0 | 256.0 | 10.0 | 1000.0 | 1000.0 | 1000.0 | 7 | COMPLETE |
| 797 | 797 | 0.634375 | 2021-03-10 17:26:13.485562 | 2021-03-10 17:26:30.506751 | 0 days 00:00:17.021189 | A D B | 256 | 128.0 | 1000.0 | 10.0 | NaN | NaN | NaN | 4 | COMPLETE |
| 798 | 798 | 0.625000 | 2021-03-10 17:26:13.498070 | 2021-03-10 17:26:30.629113 | 0 days 00:00:17.131043 | A D B | 256 | 128.0 | 1000.0 | 10.0 | 1000.0 | 1000.0 | NaN | 6 | COMPLETE |
| 799 | 799 | 0.634375 | 2021-03-10 17:26:13.525324 | 2021-03-10 17:26:30.634664 | 0 days 00:00:17.109340 | D B A | 256 | 128.0 | 1000.0 | 10.0 | 1000.0 | 1000.0 | NaN | 6 | COMPLETE |
800 rows × 15 columns
neurons_per_layer =[arch_best_params[f'neuron_l{i}'] for i in range(1,arch_best_params['num_layers']+1)]
arch_best_params['neurons_per_layer'] = neurons_per_layer
for i in range(1,arch_best_params['num_layers']+1):
if f'neuron_l{i}' in arch_best_params.keys():
del arch_best_params[f'neuron_l{i}']
else:
pass
display(Markdown("#### Modified best parameters from architecture :"))
print(arch_best_params)
{'num_layers': 6, 'arch': 'A D B', 'neurons_per_layer': [64, 256, 1000, 10, 1000, 1000]}
def objective_coarse(trial):
global arch_best_params,loss,metric,X_train,X_test,y_train,y_test
epochs = 50
lr = 0.001
# batch_size = trial.suggest_int('batch_size',1,30)
drop_rate_per_layer = [trial.suggest_uniform(f'drop_l{layers}',0.1,0.90) for layers in range(1,arch_best_params['num_layers']+1)]
activation = trial.suggest_categorical('activation',['relu','tanh'])
# n_epochs = trial.suggest_categorical('n_epochs',[100,200,500,1000])
kernel_init = trial.suggest_categorical('kernel_init', ['glorot_normal',
'glorot_uniform',
'normal',
'uniform',
'he_normal',
'he_uniform',
'lecun_normal',
'lecun_uniform'])
opts = 'adam'
if opts == 'sgd':
optimizer = tf.keras.optimizers.SGD(learning_rate=lr)
elif opts == 'adadelta':
optimizer = tf.keras.optimizers.Adadelta(learning_rate=lr)
elif opts == 'adam':
optimizer = tf.keras.optimizers.Adam(learning_rate=lr)
elif opts == 'rmsprop':
optimizer = tf.keras.optimizers.RMSprop(learning_rate=lr)
# lr = trial.suggest_categorical('lr',[1e-0, 1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6, 1e-7])
# opts = trial.suggest_categorical('opts',['sgd','adadelta','adam','rmsprop'])
# if opts == 'sgd':
# optimizer = tf.keras.optimizers.SGD(learning_rate=trial.suggest_categorical('lr',[1e-0, 1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6, 1e-7]))
# elif opts == 'adadelta':
# optimizer = tf.keras.optimizers.Adadelta(learning_rate=trial.suggest_categorical('lr',[1e-0, 1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6, 1e-7]))
# elif opts == 'adam':
# optimizer = tf.keras.optimizers.Adam(learning_rate=trial.suggest_categorical('lr',[1e-0, 1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6, 1e-7]))
# elif opts == 'rmsprop':
# optimizer = tf.keras.optimizers.RMSprop(learning_rate=trial.suggest_categorical('lr',[1e-0, 1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6, 1e-7]))
kwargs = {
# 'batch_size': batch_size,
'drop_rate_per_layer' :drop_rate_per_layer,
'activation':activation,
'n_epochs': epochs,
'kernel_init': kernel_init,
'opts': optimizer,
}
kwargs = {**arch_best_params, **kwargs}
# print(kwargs)
dnn_1 = DNN_Model('classification')
m_1 = dnn_1.create_model(X_train.shape[1],n_class,**kwargs)
# callbacks = LivePlot(2,train_loss='mean_squared_error',train_metric='mean_squared_error')
callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,['sparse_categorical_crossentropy'],['accuracy'],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
return result[1]
study_coarse = optuna.create_study(direction='maximize',sampler=optuna.samplers.TPESampler())
study_coarse.optimize(objective_coarse,n_trials=800,n_jobs=-13)
[I 2021-03-10 17:26:31,441] A new study created in memory with name: no-name-80510cf8-5186-419b-b20e-f174baef3791 [I 2021-03-10 17:27:11,790] Trial 1 finished with value: 0.45625001192092896 and parameters: {'drop_l1': 0.7114085406557237, 'drop_l2': 0.1603612157789157, 'drop_l3': 0.4003008047361315, 'drop_l4': 0.5129674739745943, 'drop_l5': 0.28051624262616226, 'drop_l6': 0.39713143876222623, 'activation': 'tanh', 'kernel_init': 'glorot_normal'}. Best is trial 1 with value: 0.45625001192092896. [I 2021-03-10 17:27:12,720] Trial 3 finished with value: 0.421875 and parameters: {'drop_l1': 0.37536965042322457, 'drop_l2': 0.7208262610408133, 'drop_l3': 0.2629647701089055, 'drop_l4': 0.18261070796069268, 'drop_l5': 0.635271356924246, 'drop_l6': 0.47418653849966486, 'activation': 'tanh', 'kernel_init': 'he_uniform'}. Best is trial 1 with value: 0.45625001192092896. [I 2021-03-10 17:27:12,774] Trial 2 finished with value: 0.5625 and parameters: {'drop_l1': 0.41426243296689735, 'drop_l2': 0.8025805963994035, 'drop_l3': 0.5010528502605506, 'drop_l4': 0.4426165575352563, 'drop_l5': 0.4811074864593766, 'drop_l6': 0.2790583083425756, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 2 with value: 0.5625. [I 2021-03-10 17:27:13,115] Trial 0 finished with value: 0.609375 and parameters: {'drop_l1': 0.5734168447058815, 'drop_l2': 0.2930423752422946, 'drop_l3': 0.7450829124471732, 'drop_l4': 0.44484937151837345, 'drop_l5': 0.5548215248942503, 'drop_l6': 0.23021223414722325, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 0 with value: 0.609375. [I 2021-03-10 17:27:50,662] Trial 4 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.4366252527539969, 'drop_l2': 0.1507035679045611, 'drop_l3': 0.18418819082791604, 'drop_l4': 0.7255751510196956, 'drop_l5': 0.29164425704472363, 'drop_l6': 0.41067545616383705, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 4 with value: 0.6343749761581421. [I 2021-03-10 17:27:52,756] Trial 5 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.40342520964317885, 'drop_l2': 0.429599816039677, 'drop_l3': 0.5898929420634361, 'drop_l4': 0.4190237781506331, 'drop_l5': 0.7039500856566571, 'drop_l6': 0.6769453185132267, 'activation': 'tanh', 'kernel_init': 'he_uniform'}. Best is trial 4 with value: 0.6343749761581421. [I 2021-03-10 17:27:53,166] Trial 6 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.13893210834062686, 'drop_l2': 0.11801293346349678, 'drop_l3': 0.35792406895735507, 'drop_l4': 0.184874380286949, 'drop_l5': 0.839113994504564, 'drop_l6': 0.44668879998533273, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:27:53,231] Trial 7 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.18033805822839677, 'drop_l2': 0.27309948804113726, 'drop_l3': 0.7693138980610109, 'drop_l4': 0.77724895227895, 'drop_l5': 0.8353941764611565, 'drop_l6': 0.5422015231603385, 'activation': 'tanh', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:28:28,674] Trial 8 finished with value: 0.625 and parameters: {'drop_l1': 0.16243056886344487, 'drop_l2': 0.1866381887702911, 'drop_l3': 0.509965310031034, 'drop_l4': 0.21715803370332695, 'drop_l5': 0.5820688834420128, 'drop_l6': 0.8057163479642818, 'activation': 'tanh', 'kernel_init': 'he_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:28:32,995] Trial 9 finished with value: 0.578125 and parameters: {'drop_l1': 0.39540603743603664, 'drop_l2': 0.4010038973342983, 'drop_l3': 0.29702043738692546, 'drop_l4': 0.5736240684749319, 'drop_l5': 0.6739827356177679, 'drop_l6': 0.3190612020135388, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:28:33,768] Trial 10 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.2558278575158228, 'drop_l2': 0.696702753780881, 'drop_l3': 0.20943395582410762, 'drop_l4': 0.6437226344807505, 'drop_l5': 0.7945820389714927, 'drop_l6': 0.3593799784216992, 'activation': 'tanh', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:28:34,053] Trial 11 finished with value: 0.578125 and parameters: {'drop_l1': 0.10779643994821148, 'drop_l2': 0.5420654163529381, 'drop_l3': 0.19506369357798228, 'drop_l4': 0.5072575297800598, 'drop_l5': 0.21532764846905314, 'drop_l6': 0.2855881157683573, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:29:04,048] Trial 12 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.4263296350768636, 'drop_l2': 0.7074023993422979, 'drop_l3': 0.6426001482379993, 'drop_l4': 0.8221240686299641, 'drop_l5': 0.819290572494883, 'drop_l6': 0.4503448820763125, 'activation': 'tanh', 'kernel_init': 'glorot_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:29:13,641] Trial 13 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.8712628978555031, 'drop_l2': 0.5550162510101927, 'drop_l3': 0.10302863238211296, 'drop_l4': 0.1148431393438561, 'drop_l5': 0.8859555244804709, 'drop_l6': 0.10190337569747426, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:29:13,646] Trial 14 finished with value: 0.5093749761581421 and parameters: {'drop_l1': 0.810109197441589, 'drop_l2': 0.14025221620190279, 'drop_l3': 0.1941804661487121, 'drop_l4': 0.88922648384957, 'drop_l5': 0.10347454878498172, 'drop_l6': 0.14732282181347278, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:29:13,676] Trial 15 finished with value: 0.421875 and parameters: {'drop_l1': 0.8901623769081646, 'drop_l2': 0.10485190490598201, 'drop_l3': 0.136218299564167, 'drop_l4': 0.834357414762116, 'drop_l5': 0.3774412375099825, 'drop_l6': 0.6063639803724448, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:29:34,721] Trial 16 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.8983255795448528, 'drop_l2': 0.15224324484054583, 'drop_l3': 0.3621847456009533, 'drop_l4': 0.3149400902572632, 'drop_l5': 0.37473214351949613, 'drop_l6': 0.642212986023809, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:29:54,680] Trial 17 finished with value: 0.5687500238418579 and parameters: {'drop_l1': 0.609318274492685, 'drop_l2': 0.10409830225274487, 'drop_l3': 0.38847677819364546, 'drop_l4': 0.3087585447363115, 'drop_l5': 0.4114918440878371, 'drop_l6': 0.7077436074631304, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:29:54,751] Trial 19 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.5828856707133926, 'drop_l2': 0.26055614849943737, 'drop_l3': 0.3662267425489233, 'drop_l4': 0.29507561513440383, 'drop_l5': 0.42896889919368014, 'drop_l6': 0.7143890388139427, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:29:54,910] Trial 18 finished with value: 0.625 and parameters: {'drop_l1': 0.5909012567500063, 'drop_l2': 0.10942587154190166, 'drop_l3': 0.3808876089049898, 'drop_l4': 0.3140133324616163, 'drop_l5': 0.4227336077455829, 'drop_l6': 0.6848365029784063, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:30:07,196] Trial 20 finished with value: 0.534375011920929 and parameters: {'drop_l1': 0.5948415101854059, 'drop_l2': 0.2768171770663712, 'drop_l3': 0.39148573203955706, 'drop_l4': 0.698568637743245, 'drop_l5': 0.1521454660370491, 'drop_l6': 0.5352606348310367, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:30:35,935] Trial 22 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.28238058303122626, 'drop_l2': 0.35887951985801614, 'drop_l3': 0.8966332156090175, 'drop_l4': 0.6781310931190704, 'drop_l5': 0.11025440806278847, 'drop_l6': 0.526680178354692, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:30:36,969] Trial 23 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.3064138600949912, 'drop_l2': 0.34976223492428427, 'drop_l3': 0.2676107891872074, 'drop_l4': 0.7187513860359812, 'drop_l5': 0.12362611507326701, 'drop_l6': 0.5540569062656336, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:30:37,498] Trial 21 finished with value: 0.528124988079071 and parameters: {'drop_l1': 0.2971438811119906, 'drop_l2': 0.2677044874673685, 'drop_l3': 0.3208241791753004, 'drop_l4': 0.6671177728679983, 'drop_l5': 0.13295982874267587, 'drop_l6': 0.5165660865743016, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:30:42,991] Trial 24 finished with value: 0.515625 and parameters: {'drop_l1': 0.2643492120647465, 'drop_l2': 0.8707126806785066, 'drop_l3': 0.22083323608299038, 'drop_l4': 0.6542617664007305, 'drop_l5': 0.7846286460421957, 'drop_l6': 0.3861123240145149, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:31:16,600] Trial 25 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.26522946698156047, 'drop_l2': 0.6922755645321943, 'drop_l3': 0.25430269087159474, 'drop_l4': 0.6357962617975454, 'drop_l5': 0.7963873777909216, 'drop_l6': 0.39250828294230095, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:31:19,478] Trial 26 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.31096546309776907, 'drop_l2': 0.23395296175684335, 'drop_l3': 0.26699243295557745, 'drop_l4': 0.7472292770114193, 'drop_l5': 0.23790083824337072, 'drop_l6': 0.42416468497087156, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:31:19,524] Trial 27 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.4993657635486101, 'drop_l2': 0.21394335497215006, 'drop_l3': 0.258961265943562, 'drop_l4': 0.7412069574535105, 'drop_l5': 0.2625111743257666, 'drop_l6': 0.40000732443356535, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:31:21,316] Trial 28 finished with value: 0.5874999761581421 and parameters: {'drop_l1': 0.49695093020236486, 'drop_l2': 0.20895333391691084, 'drop_l3': 0.10045407883089233, 'drop_l4': 0.7444084264379783, 'drop_l5': 0.2770332751273378, 'drop_l6': 0.41349083819382926, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:31:55,587] Trial 29 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.47588323371520286, 'drop_l2': 0.2185424485241114, 'drop_l3': 0.11695672590247105, 'drop_l4': 0.7570797679911109, 'drop_l5': 0.2837592779936685, 'drop_l6': 0.4372898994461433, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:32:01,102] Trial 31 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.11150533641684907, 'drop_l2': 0.33859512607940334, 'drop_l3': 0.4544216199778642, 'drop_l4': 0.5971719400547943, 'drop_l5': 0.1856068210528291, 'drop_l6': 0.5873154775259534, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:32:01,251] Trial 30 finished with value: 0.578125 and parameters: {'drop_l1': 0.18938532177825929, 'drop_l2': 0.20907628238607165, 'drop_l3': 0.4402134547575098, 'drop_l4': 0.8878962900886369, 'drop_l5': 0.1987853653773295, 'drop_l6': 0.5834451904792872, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:32:01,656] Trial 32 finished with value: 0.625 and parameters: {'drop_l1': 0.11403831005537038, 'drop_l2': 0.3553044789071813, 'drop_l3': 0.4472762775467905, 'drop_l4': 0.8837509363835716, 'drop_l5': 0.19111131855620916, 'drop_l6': 0.5862636278877026, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:32:26,413] Trial 33 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.33962631183443837, 'drop_l2': 0.3598276981142353, 'drop_l3': 0.4588754015779303, 'drop_l4': 0.8860541345842208, 'drop_l5': 0.19092704691314416, 'drop_l6': 0.1971810772904457, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:32:41,631] Trial 34 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.33619411900476026, 'drop_l2': 0.33270718847423464, 'drop_l3': 0.16585701728029373, 'drop_l4': 0.8838371239566807, 'drop_l5': 0.22293205954218415, 'drop_l6': 0.22816203857862427, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:32:41,998] Trial 35 finished with value: 0.640625 and parameters: {'drop_l1': 0.34647334886797665, 'drop_l2': 0.4552566357465644, 'drop_l3': 0.320806107250909, 'drop_l4': 0.8183317010504092, 'drop_l5': 0.24235896248207803, 'drop_l6': 0.4761861954261974, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:32:42,516] Trial 36 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.33548767175455, 'drop_l2': 0.48885155405980896, 'drop_l3': 0.1580331920573575, 'drop_l4': 0.5630372469247917, 'drop_l5': 0.3337346909114886, 'drop_l6': 0.48486765460818154, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:32:58,233] Trial 37 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.2268542720640216, 'drop_l2': 0.44765326132926764, 'drop_l3': 0.15859825568559904, 'drop_l4': 0.824145997047499, 'drop_l5': 0.3347827882235568, 'drop_l6': 0.4722705402570584, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:33:23,408] Trial 39 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.21482495784132297, 'drop_l2': 0.4647002712084565, 'drop_l3': 0.32263920873512253, 'drop_l4': 0.8311527964890436, 'drop_l5': 0.32434733496164286, 'drop_l6': 0.4822449396328098, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:33:24,812] Trial 40 finished with value: 0.609375 and parameters: {'drop_l1': 0.22073317579855853, 'drop_l2': 0.4794939023304257, 'drop_l3': 0.3142620727429744, 'drop_l4': 0.8122958880313529, 'drop_l5': 0.5207733738766933, 'drop_l6': 0.32235364874836225, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:33:24,830] Trial 38 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.2215604420636408, 'drop_l2': 0.4398702985010056, 'drop_l3': 0.32188620884619, 'drop_l4': 0.5694761844944899, 'drop_l5': 0.3261605262167928, 'drop_l6': 0.47264332958000027, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:33:33,766] Trial 41 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.22071173907492558, 'drop_l2': 0.640522963944483, 'drop_l3': 0.3057639125575181, 'drop_l4': 0.3924588153607279, 'drop_l5': 0.49536863414088805, 'drop_l6': 0.34640820150524454, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:34:03,322] Trial 42 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.15535957973964393, 'drop_l2': 0.5544495805161053, 'drop_l3': 0.5441329993195123, 'drop_l4': 0.40366554271035027, 'drop_l5': 0.5629553117821265, 'drop_l6': 0.33862456941075403, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:34:05,942] Trial 44 finished with value: 0.581250011920929 and parameters: {'drop_l1': 0.43398238263583566, 'drop_l2': 0.6086819208791443, 'drop_l3': 0.2727970189838699, 'drop_l4': 0.3924878461788544, 'drop_l5': 0.24528916013334673, 'drop_l6': 0.3453083429984645, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:34:07,306] Trial 43 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1571740241967014, 'drop_l2': 0.5604134482797468, 'drop_l3': 0.2735319411454166, 'drop_l4': 0.7171591101322995, 'drop_l5': 0.23158761804252714, 'drop_l6': 0.3455659923108285, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:34:10,383] Trial 45 finished with value: 0.5718749761581421 and parameters: {'drop_l1': 0.31949932004811205, 'drop_l2': 0.5845577731309232, 'drop_l3': 0.2785613513232698, 'drop_l4': 0.42356573606056414, 'drop_l5': 0.6057941813711993, 'drop_l6': 0.34029143627825187, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:34:40,847] Trial 46 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.45195520146329415, 'drop_l2': 0.4084405088221313, 'drop_l3': 0.2450142071487025, 'drop_l4': 0.7089287522144347, 'drop_l5': 0.24311791062319393, 'drop_l6': 0.2761882263897625, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:34:45,581] Trial 47 finished with value: 0.59375 and parameters: {'drop_l1': 0.36931526998326014, 'drop_l2': 0.606583200043805, 'drop_l3': 0.23660788643058167, 'drop_l4': 0.2279458937186757, 'drop_l5': 0.6081037210612963, 'drop_l6': 0.2914219159889371, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:34:46,381] Trial 48 finished with value: 0.45625001192092896 and parameters: {'drop_l1': 0.37603011359849037, 'drop_l2': 0.4038432083348161, 'drop_l3': 0.23479397864034987, 'drop_l4': 0.7937385774695245, 'drop_l5': 0.6203425044357216, 'drop_l6': 0.2526378126821529, 'activation': 'tanh', 'kernel_init': 'he_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:34:48,111] Trial 49 finished with value: 0.48124998807907104 and parameters: {'drop_l1': 0.3811181345366173, 'drop_l2': 0.4025787440884175, 'drop_l3': 0.23757501457041102, 'drop_l4': 0.7803170159965208, 'drop_l5': 0.15027429225623168, 'drop_l6': 0.2817488687747791, 'activation': 'tanh', 'kernel_init': 'he_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:35:15,408] Trial 50 finished with value: 0.5562499761581421 and parameters: {'drop_l1': 0.37596419795196684, 'drop_l2': 0.5168485037149101, 'drop_l3': 0.35136963150021544, 'drop_l4': 0.7743468483660804, 'drop_l5': 0.15453153098785666, 'drop_l6': 0.4301617409816905, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:35:23,718] Trial 51 finished with value: 0.48750001192092896 and parameters: {'drop_l1': 0.1549137921125332, 'drop_l2': 0.3990542891898936, 'drop_l3': 0.3545229366920227, 'drop_l4': 0.78513139473579, 'drop_l5': 0.13511265883284707, 'drop_l6': 0.557844607903724, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:35:24,806] Trial 52 finished with value: 0.559374988079071 and parameters: {'drop_l1': 0.30180894475176356, 'drop_l2': 0.6673590860438829, 'drop_l3': 0.3364217935706143, 'drop_l4': 0.47482794035864395, 'drop_l5': 0.7179307808868679, 'drop_l6': 0.44351839950135313, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:35:25,244] Trial 53 finished with value: 0.625 and parameters: {'drop_l1': 0.1357584717392973, 'drop_l2': 0.32401797026192974, 'drop_l3': 0.3461398529244522, 'drop_l4': 0.4770982107225389, 'drop_l5': 0.6959512090396139, 'drop_l6': 0.42363483380416334, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:35:44,515] Trial 54 finished with value: 0.640625 and parameters: {'drop_l1': 0.15589139993073112, 'drop_l2': 0.6602352251841581, 'drop_l3': 0.4114630229073565, 'drop_l4': 0.12589565285112078, 'drop_l5': 0.6997766574703657, 'drop_l6': 0.3687448736010326, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:36:03,324] Trial 55 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.13206583208541067, 'drop_l2': 0.6202460532888036, 'drop_l3': 0.29167515366560565, 'drop_l4': 0.23811463556234394, 'drop_l5': 0.6963545119729322, 'drop_l6': 0.3794794518629311, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:36:03,383] Trial 56 finished with value: 0.5406249761581421 and parameters: {'drop_l1': 0.13170784528152082, 'drop_l2': 0.783182339748345, 'drop_l3': 0.2940262230880776, 'drop_l4': 0.18664588181707367, 'drop_l5': 0.49675084878936904, 'drop_l6': 0.3841515332883019, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:36:04,809] Trial 57 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.18428412675177652, 'drop_l2': 0.7978476099070748, 'drop_l3': 0.28974862935853035, 'drop_l4': 0.6025086988050197, 'drop_l5': 0.5021258624418782, 'drop_l6': 0.3576705438699025, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:36:16,470] Trial 58 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.5348059381339858, 'drop_l2': 0.7753279873164016, 'drop_l3': 0.4155405089805167, 'drop_l4': 0.16964635875986656, 'drop_l5': 0.8593087487580394, 'drop_l6': 0.37910916568732844, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:36:41,780] Trial 59 finished with value: 0.640625 and parameters: {'drop_l1': 0.5449296800416927, 'drop_l2': 0.7638050095235758, 'drop_l3': 0.3939110339462324, 'drop_l4': 0.13021579593805968, 'drop_l5': 0.7407051321376359, 'drop_l6': 0.5033831484090932, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:36:43,543] Trial 60 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.1891066259933283, 'drop_l2': 0.5153693350608003, 'drop_l3': 0.4126020093415662, 'drop_l4': 0.10451650249309664, 'drop_l5': 0.8925002780004944, 'drop_l6': 0.5054251519205015, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:36:43,550] Trial 61 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.5429041721049005, 'drop_l2': 0.24364157821193583, 'drop_l3': 0.4131930597281213, 'drop_l4': 0.10793846313941498, 'drop_l5': 0.8893281469665326, 'drop_l6': 0.5084255390147705, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:36:49,650] Trial 62 finished with value: 0.640625 and parameters: {'drop_l1': 0.24706823836899627, 'drop_l2': 0.3072350256158235, 'drop_l3': 0.49786485496547617, 'drop_l4': 0.12973852917559484, 'drop_l5': 0.7188363317269352, 'drop_l6': 0.5095208919539912, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:37:21,551] Trial 63 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.6835619203181927, 'drop_l2': 0.8747909428097627, 'drop_l3': 0.5038314485668024, 'drop_l4': 0.1410350488088543, 'drop_l5': 0.7411810016843123, 'drop_l6': 0.6269096805243457, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:37:25,709] Trial 64 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.6874789287129555, 'drop_l2': 0.7364781057979785, 'drop_l3': 0.538584408638513, 'drop_l4': 0.1644523237412122, 'drop_l5': 0.7430400573776419, 'drop_l6': 0.5489165512234564, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:37:25,759] Trial 65 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.6144970267496007, 'drop_l2': 0.7430278396291282, 'drop_l3': 0.485068610432402, 'drop_l4': 0.16439920662737753, 'drop_l5': 0.7467120662739115, 'drop_l6': 0.6363787582295004, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:37:27,160] Trial 66 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.24157689784263178, 'drop_l2': 0.17051338746111916, 'drop_l3': 0.4832427500498762, 'drop_l4': 0.2577023184126316, 'drop_l5': 0.7596096975658535, 'drop_l6': 0.6335497064637544, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:37:56,001] Trial 67 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.24764406973213518, 'drop_l2': 0.2983378657138159, 'drop_l3': 0.626773249290252, 'drop_l4': 0.2521765614842956, 'drop_l5': 0.7606604335191637, 'drop_l6': 0.5647644151012358, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:38:07,411] Trial 69 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.2377994405544904, 'drop_l2': 0.844783060692864, 'drop_l3': 0.6275076609990866, 'drop_l4': 0.24350894462144465, 'drop_l5': 0.6579299647065144, 'drop_l6': 0.5498856644935239, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:38:08,710] Trial 68 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.2597500755784762, 'drop_l2': 0.12801293773227385, 'drop_l3': 0.5998921546345953, 'drop_l4': 0.25362262120711637, 'drop_l5': 0.6487169301233038, 'drop_l6': 0.5615609254789177, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:38:08,962] Trial 70 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.2752315681911907, 'drop_l2': 0.3025855977899679, 'drop_l3': 0.21243716350332434, 'drop_l4': 0.2174459107625744, 'drop_l5': 0.8454546378381604, 'drop_l6': 0.45271020813800805, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:38:26,537] Trial 71 finished with value: 0.625 and parameters: {'drop_l1': 0.10513756817036322, 'drop_l2': 0.2841853816579023, 'drop_l3': 0.21360924336410836, 'drop_l4': 0.13350006472408404, 'drop_l5': 0.6662901550347549, 'drop_l6': 0.4464277825363062, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:38:46,694] Trial 72 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.41517798397411676, 'drop_l2': 0.6500330704844217, 'drop_l3': 0.3781550534528623, 'drop_l4': 0.2064815044835321, 'drop_l5': 0.6533936399928939, 'drop_l6': 0.4590708206008321, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:38:47,701] Trial 74 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.3027804428224129, 'drop_l2': 0.6428477603821245, 'drop_l3': 0.38587075842612034, 'drop_l4': 0.13470486012658794, 'drop_l5': 0.6772467140955996, 'drop_l6': 0.4604760497746531, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:38:49,237] Trial 73 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.4691711311979611, 'drop_l2': 0.6754696894229244, 'drop_l3': 0.3844380386084969, 'drop_l4': 0.19882756093345683, 'drop_l5': 0.8289450576309405, 'drop_l6': 0.4573478394555714, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:38:59,918] Trial 75 finished with value: 0.59375 and parameters: {'drop_l1': 0.4022106443653733, 'drop_l2': 0.6439520929488545, 'drop_l3': 0.38027554829537535, 'drop_l4': 0.13373052503405228, 'drop_l5': 0.8152803479126648, 'drop_l6': 0.898611974161003, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:39:27,296] Trial 76 finished with value: 0.5874999761581421 and parameters: {'drop_l1': 0.19563314533536744, 'drop_l2': 0.37525462856144287, 'drop_l3': 0.4350230000431521, 'drop_l4': 0.1254867505297046, 'drop_l5': 0.8082609696912759, 'drop_l6': 0.8907486454123614, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:39:29,312] Trial 77 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.13212024666954336, 'drop_l2': 0.24847106703436497, 'drop_l3': 0.42784277315586206, 'drop_l4': 0.1970610168913264, 'drop_l5': 0.7800572931885796, 'drop_l6': 0.5196712706153458, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:39:30,396] Trial 78 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.20022437080612243, 'drop_l2': 0.23402965382642033, 'drop_l3': 0.4180937031488262, 'drop_l4': 0.8447342017798064, 'drop_l5': 0.7935119854899324, 'drop_l6': 0.5240417117981521, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:39:36,412] Trial 79 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.1978786039416895, 'drop_l2': 0.37495160583718773, 'drop_l3': 0.6943850888363426, 'drop_l4': 0.8603052420867914, 'drop_l5': 0.7145442389090003, 'drop_l6': 0.49283206876266417, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:40:09,577] Trial 80 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.286200359743918, 'drop_l2': 0.25503239429613644, 'drop_l3': 0.5285596274462981, 'drop_l4': 0.8593063873250424, 'drop_l5': 0.45132333642353656, 'drop_l6': 0.4079832285667189, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:40:11,851] Trial 81 finished with value: 0.609375 and parameters: {'drop_l1': 0.34993321208776135, 'drop_l2': 0.8359858176490094, 'drop_l3': 0.5366147278187509, 'drop_l4': 0.28229940660571184, 'drop_l5': 0.7179726056184325, 'drop_l6': 0.4904190921894681, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:40:12,865] Trial 82 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.6342122483438003, 'drop_l2': 0.8420292693483639, 'drop_l3': 0.5480629410916865, 'drop_l4': 0.35669432372296694, 'drop_l5': 0.7049490758848236, 'drop_l6': 0.4068407912102222, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:40:15,998] Trial 83 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.3454239673447644, 'drop_l2': 0.7150175567190745, 'drop_l3': 0.467127848092106, 'drop_l4': 0.2803841498465464, 'drop_l5': 0.6794164317982891, 'drop_l6': 0.4104604987992541, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:40:53,621] Trial 84 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.30443856089302734, 'drop_l2': 0.18870219898852586, 'drop_l3': 0.3327245139372838, 'drop_l4': 0.1449500267731723, 'drop_l5': 0.6849374155074932, 'drop_l6': 0.3701633409288293, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:40:55,259] Trial 85 finished with value: 0.609375 and parameters: {'drop_l1': 0.311705001405917, 'drop_l2': 0.7111306423205018, 'drop_l3': 0.4661457902574435, 'drop_l4': 0.14553063413253628, 'drop_l5': 0.6985532281897698, 'drop_l6': 0.4165127252412718, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:40:56,236] Trial 86 finished with value: 0.550000011920929 and parameters: {'drop_l1': 0.314790576523909, 'drop_l2': 0.7098411483563775, 'drop_l3': 0.3362950197202508, 'drop_l4': 0.148787859173721, 'drop_l5': 0.864209384656588, 'drop_l6': 0.3664891045255698, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:40:56,973] Trial 87 finished with value: 0.625 and parameters: {'drop_l1': 0.30692720331405987, 'drop_l2': 0.6169926512031274, 'drop_l3': 0.3359231882295847, 'drop_l4': 0.15387467984989747, 'drop_l5': 0.6967987561041926, 'drop_l6': 0.47110728160223725, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:41:35,024] Trial 88 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.2760923503199826, 'drop_l2': 0.30322886864099635, 'drop_l3': 0.18062195801704725, 'drop_l4': 0.2238309304507231, 'drop_l5': 0.37935995379097387, 'drop_l6': 0.43130760216273156, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:41:38,590] Trial 89 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.28120460166178984, 'drop_l2': 0.3099218757641991, 'drop_l3': 0.18825615290740738, 'drop_l4': 0.6945782330406206, 'drop_l5': 0.8407006826685565, 'drop_l6': 0.3109936280636232, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:41:39,079] Trial 90 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.1716546229373948, 'drop_l2': 0.6051450740089853, 'drop_l3': 0.36195755527454015, 'drop_l4': 0.6845736361198633, 'drop_l5': 0.6333324747629396, 'drop_l6': 0.4693825071588081, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:41:39,161] Trial 91 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.16932051851198895, 'drop_l2': 0.5784347858234227, 'drop_l3': 0.18287867174862354, 'drop_l4': 0.10016398376561886, 'drop_l5': 0.5709668072045571, 'drop_l6': 0.6047372595199906, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:42:14,695] Trial 92 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.16785614300055884, 'drop_l2': 0.6811835925185399, 'drop_l3': 0.39863117007295096, 'drop_l4': 0.6813610884811286, 'drop_l5': 0.764740842413557, 'drop_l6': 0.608067606164827, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:42:20,639] Trial 95 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.5587677493898051, 'drop_l2': 0.5639779419198855, 'drop_l3': 0.2629364180161704, 'drop_l4': 0.10239179949434359, 'drop_l5': 0.5877436923140867, 'drop_l6': 0.5351571006850269, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:42:21,620] Trial 94 finished with value: 0.5874999761581421 and parameters: {'drop_l1': 0.5653274248427504, 'drop_l2': 0.34326126121692613, 'drop_l3': 0.26407722999366856, 'drop_l4': 0.18240903009491805, 'drop_l5': 0.5641109767475478, 'drop_l6': 0.6042487789902804, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:42:21,724] Trial 93 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.3569263305560963, 'drop_l2': 0.5754088522123973, 'drop_l3': 0.2591542015937403, 'drop_l4': 0.7560476062420494, 'drop_l5': 0.8709352656306992, 'drop_l6': 0.6108588893013793, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:42:46,448] Trial 96 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.14819943990987336, 'drop_l2': 0.5300627815812121, 'drop_l3': 0.302468490793086, 'drop_l4': 0.7283819656960061, 'drop_l5': 0.5651124387802118, 'drop_l6': 0.6640686296369004, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:43:01,111] Trial 97 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.15045148342352524, 'drop_l2': 0.6833114250019844, 'drop_l3': 0.3006415321956591, 'drop_l4': 0.6554154003697348, 'drop_l5': 0.7725867688413994, 'drop_l6': 0.6001157664209625, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:43:02,801] Trial 98 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.14630544869047515, 'drop_l2': 0.688945097813779, 'drop_l3': 0.12985419885710925, 'drop_l4': 0.7487765293616274, 'drop_l5': 0.5347184636492551, 'drop_l6': 0.6649498679353041, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:43:03,014] Trial 99 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.1400543756716601, 'drop_l2': 0.523958831655025, 'drop_l3': 0.13491809598516363, 'drop_l4': 0.1174075293826395, 'drop_l5': 0.5268977173534769, 'drop_l6': 0.6844084730442017, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:43:17,571] Trial 100 finished with value: 0.609375 and parameters: {'drop_l1': 0.16639012239857193, 'drop_l2': 0.6862221682731682, 'drop_l3': 0.13098710455369605, 'drop_l4': 0.6347397095081301, 'drop_l5': 0.7773432617053039, 'drop_l6': 0.7416308594015208, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:43:39,316] Trial 101 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.3278945089833327, 'drop_l2': 0.5873763494852811, 'drop_l3': 0.36971898492056043, 'drop_l4': 0.12006410348496575, 'drop_l5': 0.1012678618821891, 'drop_l6': 0.49952603943403506, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:43:41,365] Trial 102 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.1230993408371378, 'drop_l2': 0.7351616355835711, 'drop_l3': 0.13911145015133133, 'drop_l4': 0.7951581774448163, 'drop_l5': 0.4582089437852474, 'drop_l6': 0.7338414514163909, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:43:42,839] Trial 103 finished with value: 0.640625 and parameters: {'drop_l1': 0.12207490936727895, 'drop_l2': 0.7524783488033318, 'drop_l3': 0.12091932583422342, 'drop_l4': 0.17825838397545046, 'drop_l5': 0.5411093111564842, 'drop_l6': 0.721481196044972, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:43:51,229] Trial 104 finished with value: 0.5718749761581421 and parameters: {'drop_l1': 0.12411048438422698, 'drop_l2': 0.6615700773099799, 'drop_l3': 0.8769295932185948, 'drop_l4': 0.8157221772396074, 'drop_l5': 0.10098935866146354, 'drop_l6': 0.6539909767191262, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:44:17,993] Trial 105 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.516853102509789, 'drop_l2': 0.6238470356645713, 'drop_l3': 0.21457474975607913, 'drop_l4': 0.8060140783071137, 'drop_l5': 0.17086597645843907, 'drop_l6': 0.7205786996139745, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:44:20,945] Trial 107 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.11568444711040163, 'drop_l2': 0.7736334301699718, 'drop_l3': 0.16959136646471631, 'drop_l4': 0.18166209858316854, 'drop_l5': 0.5474277356323197, 'drop_l6': 0.6612667588831179, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:44:21,847] Trial 106 finished with value: 0.609375 and parameters: {'drop_l1': 0.10596288997673117, 'drop_l2': 0.42330502728004865, 'drop_l3': 0.2869090191528575, 'drop_l4': 0.1752608824234262, 'drop_l5': 0.7359510195183926, 'drop_l6': 0.43631480679478757, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:44:26,249] Trial 108 finished with value: 0.59375 and parameters: {'drop_l1': 0.1016942067314971, 'drop_l2': 0.6236866633799972, 'drop_l3': 0.16939975158440465, 'drop_l4': 0.18165986772261303, 'drop_l5': 0.5903428226573282, 'drop_l6': 0.7563466125595456, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:44:56,805] Trial 109 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.21087191205285188, 'drop_l2': 0.7461376300826353, 'drop_l3': 0.1007936583137324, 'drop_l4': 0.17396652991512535, 'drop_l5': 0.5383865079723916, 'drop_l6': 0.8056602160790786, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:45:02,061] Trial 111 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.20536529366332956, 'drop_l2': 0.7528930997464889, 'drop_l3': 0.10624323014060043, 'drop_l4': 0.10119148890128042, 'drop_l5': 0.5262588426002217, 'drop_l6': 0.6990039632437368, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:45:02,063] Trial 110 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.10017394990130274, 'drop_l2': 0.7461612237424586, 'drop_l3': 0.10317959943494495, 'drop_l4': 0.7351312554285301, 'drop_l5': 0.5275999788095842, 'drop_l6': 0.8035314873517272, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:45:03,519] Trial 112 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.22893726559248934, 'drop_l2': 0.7622135046308952, 'drop_l3': 0.10566700557730069, 'drop_l4': 0.5296746805142378, 'drop_l5': 0.5338014374565109, 'drop_l6': 0.8005017570860544, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:45:29,296] Trial 113 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.17603115287316218, 'drop_l2': 0.8094218839644473, 'drop_l3': 0.1132548595458697, 'drop_l4': 0.5412557802898466, 'drop_l5': 0.4827596317120349, 'drop_l6': 0.3917102211232523, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:45:41,516] Trial 115 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.2303646150968142, 'drop_l2': 0.2819715756607258, 'drop_l3': 0.207153335271892, 'drop_l4': 0.2153095612330443, 'drop_l5': 0.30505773293697247, 'drop_l6': 0.3958043567144606, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:45:43,041] Trial 116 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.2692525242593038, 'drop_l2': 0.31397472144363403, 'drop_l3': 0.1975113095903985, 'drop_l4': 0.23379417928713314, 'drop_l5': 0.6063195387188486, 'drop_l6': 0.5669947056901844, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:45:43,083] Trial 114 finished with value: 0.5687500238418579 and parameters: {'drop_l1': 0.1769472024677219, 'drop_l2': 0.2719363721312399, 'drop_l3': 0.22713950645402192, 'drop_l4': 0.21689983127033075, 'drop_l5': 0.63408878503103, 'drop_l6': 0.39330445934258773, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:46:01,822] Trial 117 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.258529364033211, 'drop_l2': 0.694273634403774, 'drop_l3': 0.3156706453883762, 'drop_l4': 0.7201880560053568, 'drop_l5': 0.6367002220310769, 'drop_l6': 0.588615174671243, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:46:21,371] Trial 118 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.2558427701912369, 'drop_l2': 0.2269306830358195, 'drop_l3': 0.2236584898074066, 'drop_l4': 0.7602341675106565, 'drop_l5': 0.3038770430510306, 'drop_l6': 0.5114984778547724, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:46:23,552] Trial 120 finished with value: 0.559374988079071 and parameters: {'drop_l1': 0.24211148889290396, 'drop_l2': 0.3259123456893017, 'drop_l3': 0.20544952962426544, 'drop_l4': 0.34594227199063304, 'drop_l5': 0.2711964352111322, 'drop_l6': 0.5905570359771178, 'activation': 'tanh', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:46:25,512] Trial 119 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.2490521905638206, 'drop_l2': 0.19191356073249374, 'drop_l3': 0.14810544375309453, 'drop_l4': 0.34267179364264067, 'drop_l5': 0.20552542363905119, 'drop_l6': 0.5378623467799531, 'activation': 'tanh', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:46:36,475] Trial 121 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.23938268098065063, 'drop_l2': 0.32793968835995235, 'drop_l3': 0.19913857743443164, 'drop_l4': 0.7667885521408464, 'drop_l5': 0.20851437683936136, 'drop_l6': 0.5787444362057009, 'activation': 'tanh', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:46:59,655] Trial 122 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.14535255967771332, 'drop_l2': 0.4705286862539304, 'drop_l3': 0.2031302440475376, 'drop_l4': 0.2393992772513628, 'drop_l5': 0.21626096315871848, 'drop_l6': 0.5659432093319181, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:47:02,886] Trial 123 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.13792105590164977, 'drop_l2': 0.4940319079623266, 'drop_l3': 0.15161026593669566, 'drop_l4': 0.20330331866172294, 'drop_l5': 0.2135831032441862, 'drop_l6': 0.569618679150781, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:47:04,427] Trial 124 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.29018678176080814, 'drop_l2': 0.27461188035541295, 'drop_l3': 0.2421629478021896, 'drop_l4': 0.22866810179888364, 'drop_l5': 0.6137919614125068, 'drop_l6': 0.5668863920763574, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:47:09,910] Trial 125 finished with value: 0.609375 and parameters: {'drop_l1': 0.2914256578060118, 'drop_l2': 0.2837872528934403, 'drop_l3': 0.25186383514889527, 'drop_l4': 0.16058917020497673, 'drop_l5': 0.6136695123778716, 'drop_l6': 0.7682045784033216, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:47:37,325] Trial 126 finished with value: 0.59375 and parameters: {'drop_l1': 0.1425622894121935, 'drop_l2': 0.5004097202341056, 'drop_l3': 0.24071593132050934, 'drop_l4': 0.2394540235688164, 'drop_l5': 0.24525733486697723, 'drop_l6': 0.32329585965804275, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 6 with value: 0.6499999761581421. [I 2021-03-10 17:47:41,931] Trial 127 finished with value: 0.65625 and parameters: {'drop_l1': 0.15389939758948426, 'drop_l2': 0.4619501155021868, 'drop_l3': 0.2524530441183211, 'drop_l4': 0.26438151034985136, 'drop_l5': 0.24434102888771195, 'drop_l6': 0.6209244226137465, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:47:42,402] Trial 128 finished with value: 0.609375 and parameters: {'drop_l1': 0.15396293986519827, 'drop_l2': 0.7259140781044692, 'drop_l3': 0.24948473587562117, 'drop_l4': 0.2388742983495781, 'drop_l5': 0.2562159160083988, 'drop_l6': 0.6208219601615308, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:47:45,741] Trial 129 finished with value: 0.640625 and parameters: {'drop_l1': 0.19165452911707936, 'drop_l2': 0.3811636562599984, 'drop_l3': 0.1838348382910748, 'drop_l4': 0.23477220598919185, 'drop_l5': 0.5969281532911381, 'drop_l6': 0.616285129969377, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:48:12,068] Trial 130 finished with value: 0.609375 and parameters: {'drop_l1': 0.2685109384005569, 'drop_l2': 0.4505611904514253, 'drop_l3': 0.189148841464539, 'drop_l4': 0.2762009609897799, 'drop_l5': 0.12204593394255112, 'drop_l6': 0.6272025983225505, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:48:19,985] Trial 131 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.18887481463091454, 'drop_l2': 0.47370191295338787, 'drop_l3': 0.1948698604871047, 'drop_l4': 0.26675711201919583, 'drop_l5': 0.12304213825163543, 'drop_l6': 0.6297533681171092, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:48:20,772] Trial 132 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.21474156184016935, 'drop_l2': 0.3773666400117718, 'drop_l3': 0.18382812288052353, 'drop_l4': 0.278398927924978, 'drop_l5': 0.12808831680369187, 'drop_l6': 0.5501572220880484, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:48:22,729] Trial 133 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.2149138178781142, 'drop_l2': 0.377036817851063, 'drop_l3': 0.1948560452487812, 'drop_l4': 0.28212982254920077, 'drop_l5': 0.5699856474145295, 'drop_l6': 0.6490152679417962, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:48:44,412] Trial 134 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.12040903539172994, 'drop_l2': 0.5417218808245824, 'drop_l3': 0.2831817129432513, 'drop_l4': 0.3063297542181304, 'drop_l5': 0.28964818088425043, 'drop_l6': 0.6442513643047101, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:48:58,996] Trial 135 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.20663888831880714, 'drop_l2': 0.38029861239817925, 'drop_l3': 0.27568502428849984, 'drop_l4': 0.2975016562369686, 'drop_l5': 0.582940540221783, 'drop_l6': 0.5475502959769457, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:48:59,658] Trial 136 finished with value: 0.625 and parameters: {'drop_l1': 0.33094474175257504, 'drop_l2': 0.46103825331685105, 'drop_l3': 0.2815885665359193, 'drop_l4': 0.3033307841072824, 'drop_l5': 0.285465177376592, 'drop_l6': 0.5703673138937763, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:49:00,872] Trial 137 finished with value: 0.59375 and parameters: {'drop_l1': 0.16091629093257132, 'drop_l2': 0.4470089449667971, 'drop_l3': 0.2728968905609426, 'drop_l4': 0.22166558260337893, 'drop_l5': 0.30603832718732393, 'drop_l6': 0.5343218808677836, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:49:14,580] Trial 138 finished with value: 0.5843750238418579 and parameters: {'drop_l1': 0.829426722164093, 'drop_l2': 0.4321516478029672, 'drop_l3': 0.27233134855233765, 'drop_l4': 0.23055612586710633, 'drop_l5': 0.17611993585762376, 'drop_l6': 0.5740551176417141, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:49:39,641] Trial 139 finished with value: 0.5874999761581421 and parameters: {'drop_l1': 0.8379310126690771, 'drop_l2': 0.46001600277777693, 'drop_l3': 0.2251289695687795, 'drop_l4': 0.22711952927549858, 'drop_l5': 0.1720140046989857, 'drop_l6': 0.5720315664726591, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:49:40,842] Trial 140 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.1881481022473922, 'drop_l2': 0.4246483849739254, 'drop_l3': 0.2310815205209198, 'drop_l4': 0.2151190067341193, 'drop_l5': 0.18221124176535175, 'drop_l6': 0.37625981982137513, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:49:41,015] Trial 141 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.1785511315397511, 'drop_l2': 0.4240788516842924, 'drop_l3': 0.23430483651203024, 'drop_l4': 0.19246709733465578, 'drop_l5': 0.5109061775861908, 'drop_l6': 0.6727788905987191, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:49:47,948] Trial 142 finished with value: 0.581250011920929 and parameters: {'drop_l1': 0.1852717812969919, 'drop_l2': 0.31559944693654435, 'drop_l3': 0.227685813989893, 'drop_l4': 0.2573255071462847, 'drop_l5': 0.6001040238290162, 'drop_l6': 0.6976507926740916, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:50:21,025] Trial 143 finished with value: 0.640625 and parameters: {'drop_l1': 0.1798325842977866, 'drop_l2': 0.3557514910762006, 'drop_l3': 0.17379308458773185, 'drop_l4': 0.2533547900627857, 'drop_l5': 0.6003532045949092, 'drop_l6': 0.5925086495463159, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:50:22,066] Trial 145 finished with value: 0.59375 and parameters: {'drop_l1': 0.13030035447671806, 'drop_l2': 0.2626387288264447, 'drop_l3': 0.12622940626871285, 'drop_l4': 0.20005232875099999, 'drop_l5': 0.5503125728753235, 'drop_l6': 0.35326677840554727, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:50:23,697] Trial 144 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.23137603274467283, 'drop_l2': 0.359600961839629, 'drop_l3': 0.12382762608490379, 'drop_l4': 0.19625992625530314, 'drop_l5': 0.6027924545670792, 'drop_l6': 0.6798127760106417, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:50:25,327] Trial 146 finished with value: 0.515625 and parameters: {'drop_l1': 0.13284388485449375, 'drop_l2': 0.356046636737572, 'drop_l3': 0.16846424650421962, 'drop_l4': 0.20788603471436237, 'drop_l5': 0.5469109271870246, 'drop_l6': 0.3561031741634707, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:51:02,225] Trial 147 finished with value: 0.65625 and parameters: {'drop_l1': 0.13377337300833486, 'drop_l2': 0.34973713763205455, 'drop_l3': 0.16662100546354675, 'drop_l4': 0.20628080507833754, 'drop_l5': 0.575391222909984, 'drop_l6': 0.5885219208709529, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:51:04,059] Trial 148 finished with value: 0.5843750238418579 and parameters: {'drop_l1': 0.16127732569946673, 'drop_l2': 0.28865474326534923, 'drop_l3': 0.20893024076490319, 'drop_l4': 0.24575531241693233, 'drop_l5': 0.2289338137683664, 'drop_l6': 0.59943688211997, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:51:04,320] Trial 149 finished with value: 0.625 and parameters: {'drop_l1': 0.15597347633032438, 'drop_l2': 0.2833003974108968, 'drop_l3': 0.16066541629631656, 'drop_l4': 0.2454550381121714, 'drop_l5': 0.5771097833829978, 'drop_l6': 0.42450690102308697, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:51:04,521] Trial 150 finished with value: 0.625 and parameters: {'drop_l1': 0.15217139996734602, 'drop_l2': 0.29705334273733097, 'drop_l3': 0.21064814413963717, 'drop_l4': 0.12323840740218145, 'drop_l5': 0.7328704334754323, 'drop_l6': 0.6136394518788257, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:51:41,139] Trial 151 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.1574770964991786, 'drop_l2': 0.34484239004359407, 'drop_l3': 0.15919565590249107, 'drop_l4': 0.3239674668013313, 'drop_l5': 0.5713394671802358, 'drop_l6': 0.6147387449857104, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:51:43,904] Trial 152 finished with value: 0.581250011920929 and parameters: {'drop_l1': 0.520695572504494, 'drop_l2': 0.34634631300844626, 'drop_l3': 0.15092319084179176, 'drop_l4': 0.23074814669663318, 'drop_l5': 0.6211229803417688, 'drop_l6': 0.6142163109585528, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 127 with value: 0.65625. [I 2021-03-10 17:51:43,970] Trial 154 finished with value: 0.6812499761581421 and parameters: {'drop_l1': 0.11769241744475195, 'drop_l2': 0.1203918426967813, 'drop_l3': 0.3079277845699331, 'drop_l4': 0.16249139281479208, 'drop_l5': 0.3503972703488872, 'drop_l6': 0.5575415438784195, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:51:44,003] Trial 153 finished with value: 0.59375 and parameters: {'drop_l1': 0.11413538516221125, 'drop_l2': 0.23859700396232944, 'drop_l3': 0.31823888347066925, 'drop_l4': 0.21086635993334246, 'drop_l5': 0.262383688953494, 'drop_l6': 0.4910710834542099, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:52:18,907] Trial 155 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.3217715956989696, 'drop_l2': 0.480539946293286, 'drop_l3': 0.31765494646623355, 'drop_l4': 0.16044584560377326, 'drop_l5': 0.2601666036571129, 'drop_l6': 0.4835734556201774, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:52:23,744] Trial 157 finished with value: 0.65625 and parameters: {'drop_l1': 0.10327173779713526, 'drop_l2': 0.12657185897970105, 'drop_l3': 0.34985244919812303, 'drop_l4': 0.15308753524633983, 'drop_l5': 0.22933928361353328, 'drop_l6': 0.5600906672821875, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:52:24,760] Trial 156 finished with value: 0.640625 and parameters: {'drop_l1': 0.10253264308466176, 'drop_l2': 0.1340035117837497, 'drop_l3': 0.1775597600304502, 'drop_l4': 0.21447949944617012, 'drop_l5': 0.6238014398189883, 'drop_l6': 0.5239764934712109, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:52:25,125] Trial 158 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.31419091789357656, 'drop_l2': 0.1037983436537263, 'drop_l3': 0.30248076733086804, 'drop_l4': 0.1526131071416384, 'drop_l5': 0.23243693868454293, 'drop_l6': 0.5593883664998247, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:52:53,159] Trial 159 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.1010276256875251, 'drop_l2': 0.13690563033774378, 'drop_l3': 0.2957935905568411, 'drop_l4': 0.26448701144708964, 'drop_l5': 0.31069790020628074, 'drop_l6': 0.5598418563603814, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:53:06,909] Trial 161 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.2901549750217745, 'drop_l2': 0.16072789453795283, 'drop_l3': 0.34773141493754955, 'drop_l4': 0.1724452123814845, 'drop_l5': 0.38666254651490917, 'drop_l6': 0.5806399239357724, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:53:06,942] Trial 162 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.2797787472774975, 'drop_l2': 0.1598279053819208, 'drop_l3': 0.34699037290953766, 'drop_l4': 0.7385838186403527, 'drop_l5': 0.37222866642763547, 'drop_l6': 0.583579533606319, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:53:06,973] Trial 160 finished with value: 0.609375 and parameters: {'drop_l1': 0.28724448198456937, 'drop_l2': 0.1178321209466488, 'drop_l3': 0.30028761494132833, 'drop_l4': 0.14402067364499777, 'drop_l5': 0.3699167438787095, 'drop_l6': 0.5573957932321408, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:53:23,818] Trial 163 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.14338614679491765, 'drop_l2': 0.11578443849533344, 'drop_l3': 0.2561984524089634, 'drop_l4': 0.7418456883017853, 'drop_l5': 0.15536239796566814, 'drop_l6': 0.5859604008698958, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:53:52,933] Trial 166 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.11921826132379176, 'drop_l2': 0.5953508499707819, 'drop_l3': 0.24369023760007244, 'drop_l4': 0.18857633989490757, 'drop_l5': 0.5607378410690358, 'drop_l6': 0.5956088064331959, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:53:54,143] Trial 164 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.12015179853160304, 'drop_l2': 0.11805783781434237, 'drop_l3': 0.25469193818945973, 'drop_l4': 0.18476536049937775, 'drop_l5': 0.5143195104865395, 'drop_l6': 0.5956747556206641, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:53:54,874] Trial 165 finished with value: 0.65625 and parameters: {'drop_l1': 0.1250520506237345, 'drop_l2': 0.12286664772257386, 'drop_l3': 0.14160853674296428, 'drop_l4': 0.19030564156629257, 'drop_l5': 0.35211073975496704, 'drop_l6': 0.5972512780476138, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:54:06,372] Trial 167 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.10387219931707893, 'drop_l2': 0.20391716641134022, 'drop_l3': 0.24927039529322464, 'drop_l4': 0.19329487450970897, 'drop_l5': 0.22408097185167192, 'drop_l6': 0.5269650458568286, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:54:48,558] Trial 168 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.11932294188120075, 'drop_l2': 0.5909978920934047, 'drop_l3': 0.24559959832819564, 'drop_l4': 0.19279434402368792, 'drop_l5': 0.5584549931108829, 'drop_l6': 0.6378807777618287, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:54:51,913] Trial 169 finished with value: 0.640625 and parameters: {'drop_l1': 0.1277671656869504, 'drop_l2': 0.5985149894193631, 'drop_l3': 0.24862337703748538, 'drop_l4': 0.16995262698272418, 'drop_l5': 0.3420175044961513, 'drop_l6': 0.5424903238274109, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:54:52,168] Trial 170 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.10028204499074496, 'drop_l2': 0.1381432573002954, 'drop_l3': 0.24421703353132157, 'drop_l4': 0.20784511527679486, 'drop_l5': 0.329933737376364, 'drop_l6': 0.5354918230038549, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:54:58,947] Trial 171 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.1003165755766955, 'drop_l2': 0.5938189562081525, 'drop_l3': 0.2439340573259548, 'drop_l4': 0.19848940788645505, 'drop_l5': 0.19919586433397646, 'drop_l6': 0.5287243824168465, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:55:43,659] Trial 172 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.1379834258996559, 'drop_l2': 0.20434343209102196, 'drop_l3': 0.2423133050608589, 'drop_l4': 0.7056583403961998, 'drop_l5': 0.21974522987069828, 'drop_l6': 0.5320438913830442, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:55:49,430] Trial 173 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.1010456147606947, 'drop_l2': 0.21081809956792383, 'drop_l3': 0.2095773877433026, 'drop_l4': 0.19656012233407533, 'drop_l5': 0.3502323211652903, 'drop_l6': 0.5151982410576982, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:55:49,519] Trial 174 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.13789903192621555, 'drop_l2': 0.18212874390901213, 'drop_l3': 0.2010096100019166, 'drop_l4': 0.15830805783663293, 'drop_l5': 0.3442479216975306, 'drop_l6': 0.525717743332631, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:55:52,016] Trial 175 finished with value: 0.640625 and parameters: {'drop_l1': 0.14096255907997565, 'drop_l2': 0.21861368151812188, 'drop_l3': 0.20946742014290196, 'drop_l4': 0.15917247648564703, 'drop_l5': 0.4050239161831165, 'drop_l6': 0.5149260432149014, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:56:35,737] Trial 176 finished with value: 0.609375 and parameters: {'drop_l1': 0.14123917973565697, 'drop_l2': 0.27214958747496354, 'drop_l3': 0.2041074004520918, 'drop_l4': 0.16301526115847298, 'drop_l5': 0.3556983495863748, 'drop_l6': 0.5093898846331963, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:56:44,788] Trial 177 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.14063879755885228, 'drop_l2': 0.18184022436185326, 'drop_l3': 0.27130505798890997, 'drop_l4': 0.10286562188626479, 'drop_l5': 0.39574376840776454, 'drop_l6': 0.5602390722933401, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:56:47,236] Trial 179 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.3597223443391423, 'drop_l2': 0.2530554226710704, 'drop_l3': 0.28572806281235297, 'drop_l4': 0.2299247812552608, 'drop_l5': 0.21455640154014957, 'drop_l6': 0.5601822607203154, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:56:47,540] Trial 178 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.16633739138158812, 'drop_l2': 0.10058520008218375, 'drop_l3': 0.5736434949142966, 'drop_l4': 0.10379487417699737, 'drop_l5': 0.24199140579480224, 'drop_l6': 0.5650202125552174, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:57:22,734] Trial 180 finished with value: 0.6625000238418579 and parameters: {'drop_l1': 0.1159353506642477, 'drop_l2': 0.244541146626373, 'drop_l3': 0.2642512381841342, 'drop_l4': 0.22569451607943475, 'drop_l5': 0.40076392484040746, 'drop_l6': 0.5653951707922992, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:57:41,085] Trial 181 finished with value: 0.625 and parameters: {'drop_l1': 0.11617749016756336, 'drop_l2': 0.5546681639703824, 'drop_l3': 0.28788485851938694, 'drop_l4': 0.22483853225024483, 'drop_l5': 0.3193161430417952, 'drop_l6': 0.5476900091615, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:57:43,901] Trial 182 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.16369180315553114, 'drop_l2': 0.10092676956695903, 'drop_l3': 0.3338552706340205, 'drop_l4': 0.13749204896486247, 'drop_l5': 0.245265485009146, 'drop_l6': 0.4033514988340981, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:57:45,011] Trial 183 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.1734125130887778, 'drop_l2': 0.14568069358227065, 'drop_l3': 0.1751058335423618, 'drop_l4': 0.2569874418126835, 'drop_l5': 0.2776266927754189, 'drop_l6': 0.5931084781913789, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:58:09,806] Trial 184 finished with value: 0.609375 and parameters: {'drop_l1': 0.11397254809153874, 'drop_l2': 0.5656513925320599, 'drop_l3': 0.1439342710190992, 'drop_l4': 0.24633557416006255, 'drop_l5': 0.5808697031412845, 'drop_l6': 0.5984240491927784, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:58:35,163] Trial 185 finished with value: 0.5874999761581421 and parameters: {'drop_l1': 0.12340960617543476, 'drop_l2': 0.576720091999616, 'drop_l3': 0.266133626874042, 'drop_l4': 0.18352364953944694, 'drop_l5': 0.44450084126003503, 'drop_l6': 0.5439068397507328, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:58:39,471] Trial 186 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.11490935781119824, 'drop_l2': 0.6577519486759377, 'drop_l3': 0.26317781986870914, 'drop_l4': 0.18337318318708712, 'drop_l5': 0.27463667814835435, 'drop_l6': 0.6007346469043575, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:58:40,854] Trial 187 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.11335816734865987, 'drop_l2': 0.570365631294224, 'drop_l3': 0.14065802832053537, 'drop_l4': 0.24248305143950033, 'drop_l5': 0.5834807806819486, 'drop_l6': 0.578923558040642, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:58:59,126] Trial 188 finished with value: 0.609375 and parameters: {'drop_l1': 0.10967111050673445, 'drop_l2': 0.12687108313699105, 'drop_l3': 0.26233610961777826, 'drop_l4': 0.2128622656432687, 'drop_l5': 0.625608911848571, 'drop_l6': 0.5840917074511173, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:59:29,899] Trial 189 finished with value: 0.6812499761581421 and parameters: {'drop_l1': 0.10143410930683414, 'drop_l2': 0.15005432568323301, 'drop_l3': 0.22540304749276124, 'drop_l4': 0.2121103941185637, 'drop_l5': 0.6306478133260724, 'drop_l6': 0.577731263164556, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:59:33,807] Trial 190 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.25061097717285064, 'drop_l2': 0.2639212723527681, 'drop_l3': 0.22817917891893602, 'drop_l4': 0.21256456549518263, 'drop_l5': 0.19132467159025646, 'drop_l6': 0.5763533460932747, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:59:35,982] Trial 191 finished with value: 0.640625 and parameters: {'drop_l1': 0.14745022116041884, 'drop_l2': 0.25058506399479247, 'drop_l3': 0.21853236579322646, 'drop_l4': 0.20915356497541157, 'drop_l5': 0.4043049811956591, 'drop_l6': 0.5750613262211562, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 17:59:48,857] Trial 192 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.10117611218493047, 'drop_l2': 0.23912045173079607, 'drop_l3': 0.22921504088934408, 'drop_l4': 0.21186933472592945, 'drop_l5': 0.1950428443291607, 'drop_l6': 0.5707856986308186, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:00:25,015] Trial 193 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.10202338177727711, 'drop_l2': 0.2292164535769317, 'drop_l3': 0.21846941764443273, 'drop_l4': 0.21798276318690912, 'drop_l5': 0.6112723064503692, 'drop_l6': 0.575408835728859, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:00:30,161] Trial 194 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.10098818971760178, 'drop_l2': 0.3194101811638964, 'drop_l3': 0.21955247005050318, 'drop_l4': 0.2244505596509193, 'drop_l5': 0.63570096262372, 'drop_l6': 0.5505928312640899, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:00:32,313] Trial 195 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.2672489379841498, 'drop_l2': 0.15014191154640996, 'drop_l3': 0.31751573084872026, 'drop_l4': 0.8497004499236731, 'drop_l5': 0.2976429727292418, 'drop_l6': 0.6264653996820317, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:00:41,550] Trial 196 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.2665273663638755, 'drop_l2': 0.49892402222723015, 'drop_l3': 0.3052910331627818, 'drop_l4': 0.8418048644339946, 'drop_l5': 0.22795709526529026, 'drop_l6': 0.4425743563246056, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:01:22,187] Trial 197 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.4796247863203016, 'drop_l2': 0.6263359697438402, 'drop_l3': 0.3128471731635627, 'drop_l4': 0.45179926604672865, 'drop_l5': 0.6565016610604069, 'drop_l6': 0.625682432341703, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:01:28,062] Trial 198 finished with value: 0.518750011920929 and parameters: {'drop_l1': 0.12990361103546696, 'drop_l2': 0.1597338626787919, 'drop_l3': 0.3225865694239111, 'drop_l4': 0.8452617255033751, 'drop_l5': 0.23260578408005936, 'drop_l6': 0.45310983910665176, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:01:30,531] Trial 199 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.13511220896985177, 'drop_l2': 0.22033666539537541, 'drop_l3': 0.19254982783058042, 'drop_l4': 0.1712707862004644, 'drop_l5': 0.4653352441026933, 'drop_l6': 0.6061810704299071, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:01:35,949] Trial 200 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.12681264207131931, 'drop_l2': 0.30160210214758626, 'drop_l3': 0.18895189533192103, 'drop_l4': 0.19064565899799485, 'drop_l5': 0.5552643240819621, 'drop_l6': 0.6089105822312313, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:02:14,570] Trial 201 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.15113609305121356, 'drop_l2': 0.6191354035792664, 'drop_l3': 0.18759050631512833, 'drop_l4': 0.4867600146575298, 'drop_l5': 0.6605497392596794, 'drop_l6': 0.6399661531430841, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:02:25,576] Trial 202 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.13055088912020804, 'drop_l2': 0.6167763910319428, 'drop_l3': 0.1891057399857303, 'drop_l4': 0.42460111393291655, 'drop_l5': 0.6533707499021414, 'drop_l6': 0.6414835712723724, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:02:28,023] Trial 203 finished with value: 0.581250011920929 and parameters: {'drop_l1': 0.4684151948124705, 'drop_l2': 0.6406784675453574, 'drop_l3': 0.2855742608611323, 'drop_l4': 0.265916920957249, 'drop_l5': 0.4254569817669766, 'drop_l6': 0.6319490027318755, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:02:31,107] Trial 204 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1526054274766408, 'drop_l2': 0.6271113738911239, 'drop_l3': 0.2862045165590944, 'drop_l4': 0.4761155985089248, 'drop_l5': 0.6492168422518158, 'drop_l6': 0.6352547776945964, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:03:02,710] Trial 205 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.4528942031438213, 'drop_l2': 0.30710340275336817, 'drop_l3': 0.28140935840446185, 'drop_l4': 0.18610136306408204, 'drop_l5': 0.6493386354936161, 'drop_l6': 0.6607438918488276, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:03:16,364] Trial 206 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.12600349384788045, 'drop_l2': 0.6355317237049283, 'drop_l3': 0.16604364879228312, 'drop_l4': 0.1978722858036886, 'drop_l5': 0.5693698862554192, 'drop_l6': 0.6613555712147706, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:03:18,964] Trial 207 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.12447830316769426, 'drop_l2': 0.6186156877338422, 'drop_l3': 0.15852801892363683, 'drop_l4': 0.20111009907039595, 'drop_l5': 0.6702676812168216, 'drop_l6': 0.6699888806751478, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:03:20,635] Trial 208 finished with value: 0.5843750238418579 and parameters: {'drop_l1': 0.13086273949039384, 'drop_l2': 0.29905709845213013, 'drop_l3': 0.1622324888352937, 'drop_l4': 0.438402434901399, 'drop_l5': 0.5693833684720792, 'drop_l6': 0.6162423326282821, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:03:43,164] Trial 209 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.12356528253254585, 'drop_l2': 0.6307589829105931, 'drop_l3': 0.16459935801275627, 'drop_l4': 0.508618851557337, 'drop_l5': 0.5611112369414112, 'drop_l6': 0.6158898292029771, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:04:04,990] Trial 210 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.12690688944091466, 'drop_l2': 0.6305320043617215, 'drop_l3': 0.1538958766006334, 'drop_l4': 0.40671934214487765, 'drop_l5': 0.5414693650130904, 'drop_l6': 0.6556979337982485, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:04:08,216] Trial 212 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.12022166575043067, 'drop_l2': 0.6404134398512855, 'drop_l3': 0.1864158525845716, 'drop_l4': 0.3892909951852088, 'drop_l5': 0.5979951440750789, 'drop_l6': 0.6143372060230695, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:04:08,810] Trial 211 finished with value: 0.640625 and parameters: {'drop_l1': 0.2988906472271011, 'drop_l2': 0.6049905937937212, 'drop_l3': 0.17856919376249913, 'drop_l4': 0.4565955841893682, 'drop_l5': 0.5624877840131604, 'drop_l6': 0.6211803081901264, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:04:23,699] Trial 213 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.16814874944369065, 'drop_l2': 0.11563650874646932, 'drop_l3': 0.1790831097269061, 'drop_l4': 0.19206950719833635, 'drop_l5': 0.5436882714127121, 'drop_l6': 0.6570782124052428, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:04:52,567] Trial 214 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1690022775322098, 'drop_l2': 0.27787888775006964, 'drop_l3': 0.18694501088218224, 'drop_l4': 0.4578091945667654, 'drop_l5': 0.5897259872986599, 'drop_l6': 0.6032188908470147, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:04:56,378] Trial 215 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.16273086450928725, 'drop_l2': 0.6135390848507305, 'drop_l3': 0.19440323811441387, 'drop_l4': 0.19040387409431106, 'drop_l5': 0.5525874174217877, 'drop_l6': 0.6437988823077109, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:04:56,903] Trial 216 finished with value: 0.609375 and parameters: {'drop_l1': 0.17000595496200738, 'drop_l2': 0.26926328823396467, 'drop_l3': 0.19637034770221629, 'drop_l4': 0.6017245309712019, 'drop_l5': 0.5875409849262245, 'drop_l6': 0.6515844638998503, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:05:05,920] Trial 217 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.16361939743740492, 'drop_l2': 0.1124975633801413, 'drop_l3': 0.19595410412261116, 'drop_l4': 0.23811352468147526, 'drop_l5': 0.5499793354163167, 'drop_l6': 0.6814322075269269, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:05:38,112] Trial 218 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.14948052296608277, 'drop_l2': 0.11002105567096061, 'drop_l3': 0.1345798828556246, 'drop_l4': 0.18910465470674076, 'drop_l5': 0.5494348651511451, 'drop_l6': 0.6495010375304181, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:05:45,504] Trial 220 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.10688609532859848, 'drop_l2': 0.11910254740483339, 'drop_l3': 0.13722660965730243, 'drop_l4': 0.17316826934491597, 'drop_l5': 0.5386436551749546, 'drop_l6': 0.6890558872956922, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:05:45,515] Trial 219 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.10221014247102032, 'drop_l2': 0.20021602256020155, 'drop_l3': 0.23117786924383293, 'drop_l4': 0.1932982269654605, 'drop_l5': 0.362615878049567, 'drop_l6': 0.38809118558472755, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:05:48,591] Trial 221 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.10645062450615846, 'drop_l2': 0.125303605846619, 'drop_l3': 0.23721487319092488, 'drop_l4': 0.191800059348385, 'drop_l5': 0.6133339977007807, 'drop_l6': 0.3903601927961543, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:06:19,445] Trial 222 finished with value: 0.559374988079071 and parameters: {'drop_l1': 0.6448688769422348, 'drop_l2': 0.1307437314266994, 'drop_l3': 0.13895251833020264, 'drop_l4': 0.17737204313556407, 'drop_l5': 0.609630053178429, 'drop_l6': 0.39201430707953366, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:06:34,067] Trial 224 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.1478270675293463, 'drop_l2': 0.10064373776022419, 'drop_l3': 0.14392273762386584, 'drop_l4': 0.17341074115637037, 'drop_l5': 0.5240761844635671, 'drop_l6': 0.6720476910652525, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:06:34,227] Trial 223 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.1466966701520699, 'drop_l2': 0.10137855437618455, 'drop_l3': 0.14967525000237333, 'drop_l4': 0.17534426056095404, 'drop_l5': 0.5009153914106569, 'drop_l6': 0.7042347642064903, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:06:35,122] Trial 225 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.14448121559082863, 'drop_l2': 0.12949282092392125, 'drop_l3': 0.13878994357466384, 'drop_l4': 0.17143012360879703, 'drop_l5': 0.5197795738202279, 'drop_l6': 0.6971763782525973, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:06:58,496] Trial 226 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.10282454641945561, 'drop_l2': 0.10227105065486491, 'drop_l3': 0.17319877632276992, 'drop_l4': 0.17181373305562753, 'drop_l5': 0.5711091787583438, 'drop_l6': 0.41478988441802683, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:07:24,967] Trial 229 finished with value: 0.65625 and parameters: {'drop_l1': 0.11244080881991178, 'drop_l2': 0.10117776282926515, 'drop_l3': 0.17094423121264452, 'drop_l4': 0.14349114414535957, 'drop_l5': 0.5306084117523885, 'drop_l6': 0.6665318966202667, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:07:25,245] Trial 228 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.12987555963548914, 'drop_l2': 0.12164842726175398, 'drop_l3': 0.13167433681427912, 'drop_l4': 0.1495124258486375, 'drop_l5': 0.5191750006734674, 'drop_l6': 0.6910429573164169, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:07:26,304] Trial 227 finished with value: 0.640625 and parameters: {'drop_l1': 0.10347910751486886, 'drop_l2': 0.11424198350616878, 'drop_l3': 0.1311511378152127, 'drop_l4': 0.1722650913712001, 'drop_l5': 0.48932614034271826, 'drop_l6': 0.6893497633147796, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:07:40,587] Trial 230 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.11235805276020353, 'drop_l2': 0.1164461334213528, 'drop_l3': 0.12217135050719523, 'drop_l4': 0.15207684143224173, 'drop_l5': 0.48670858576605647, 'drop_l6': 0.6872898313505603, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:08:17,545] Trial 233 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.11284092140928978, 'drop_l2': 0.14458448870959006, 'drop_l3': 0.1722090221634302, 'drop_l4': 0.1514971856486712, 'drop_l5': 0.5327929495265539, 'drop_l6': 0.6694035607498727, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:08:19,182] Trial 231 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.1151909472890959, 'drop_l2': 0.10910559229845287, 'drop_l3': 0.16891332062164835, 'drop_l4': 0.1508775391138891, 'drop_l5': 0.4893307425505924, 'drop_l6': 0.655442659240541, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:08:19,287] Trial 232 finished with value: 0.565625011920929 and parameters: {'drop_l1': 0.11378375251367391, 'drop_l2': 0.1706810163235969, 'drop_l3': 0.1161807486487437, 'drop_l4': 0.1496320657578209, 'drop_l5': 0.5431964034162801, 'drop_l6': 0.671298245559265, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:08:24,205] Trial 234 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.13408340929347473, 'drop_l2': 0.10104506562279113, 'drop_l3': 0.16534835483176974, 'drop_l4': 0.3736516655234674, 'drop_l5': 0.5414822728427514, 'drop_l6': 0.6671664061201267, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:09:11,539] Trial 235 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.11910491464611907, 'drop_l2': 0.14776366521615905, 'drop_l3': 0.16774259521730486, 'drop_l4': 0.13122267318109826, 'drop_l5': 0.5337259126470081, 'drop_l6': 0.673357074684683, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:09:11,832] Trial 237 finished with value: 0.609375 and parameters: {'drop_l1': 0.1313798318796189, 'drop_l2': 0.15152034149000443, 'drop_l3': 0.16170863559942292, 'drop_l4': 0.19546049654950395, 'drop_l5': 0.5325447308999206, 'drop_l6': 0.6486420203230577, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:09:12,553] Trial 236 finished with value: 0.640625 and parameters: {'drop_l1': 0.13463675862642654, 'drop_l2': 0.14349367630425192, 'drop_l3': 0.15362515583960185, 'drop_l4': 0.12919484700076328, 'drop_l5': 0.5284820338590741, 'drop_l6': 0.6695995005586578, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:09:12,579] Trial 238 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.1310311691850105, 'drop_l2': 0.14137684286053914, 'drop_l3': 0.14881820156629577, 'drop_l4': 0.19240061050310647, 'drop_l5': 0.5088767482080422, 'drop_l6': 0.6490691090416264, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:10:07,485] Trial 241 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.100151591795584, 'drop_l2': 0.13652806665142544, 'drop_l3': 0.17521299219952433, 'drop_l4': 0.11297846133838367, 'drop_l5': 0.5085910506356282, 'drop_l6': 0.6420418650446321, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:10:07,589] Trial 242 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.14963993790520563, 'drop_l2': 0.1326722176612095, 'drop_l3': 0.14933833847580333, 'drop_l4': 0.14049334576088093, 'drop_l5': 0.5244426645017389, 'drop_l6': 0.7175916909303032, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:10:07,693] Trial 239 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.12994963786061248, 'drop_l2': 0.14365638067292652, 'drop_l3': 0.1767399258353839, 'drop_l4': 0.12878378892034356, 'drop_l5': 0.528757567983605, 'drop_l6': 0.7130808169494285, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:10:07,770] Trial 240 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.15199133643674434, 'drop_l2': 0.13983657015804762, 'drop_l3': 0.14674252566909698, 'drop_l4': 0.13316506687444554, 'drop_l5': 0.5206077851416459, 'drop_l6': 0.7161189445430163, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:11:06,200] Trial 243 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.1516983469020657, 'drop_l2': 0.1695136303163392, 'drop_l3': 0.14702851503400052, 'drop_l4': 0.12053285755734061, 'drop_l5': 0.5012292552554292, 'drop_l6': 0.6774666940402021, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:11:06,949] Trial 246 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.10021947501520281, 'drop_l2': 0.17357899436320315, 'drop_l3': 0.17526081529319076, 'drop_l4': 0.107718047148595, 'drop_l5': 0.5164853327834171, 'drop_l6': 0.646781545571899, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:11:07,810] Trial 244 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.1026120964370051, 'drop_l2': 0.15984039220029977, 'drop_l3': 0.1693672851754176, 'drop_l4': 0.1143558302146342, 'drop_l5': 0.5050428345595996, 'drop_l6': 0.6454611821281947, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:11:07,843] Trial 245 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.10083872187475901, 'drop_l2': 0.1261873904238487, 'drop_l3': 0.179633784316239, 'drop_l4': 0.1594022556711467, 'drop_l5': 0.5116766661572832, 'drop_l6': 0.6449387613392888, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:12:02,684] Trial 247 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.1000739740267916, 'drop_l2': 0.12268681856916215, 'drop_l3': 0.1781017728670206, 'drop_l4': 0.16306315522640608, 'drop_l5': 0.5602294144912743, 'drop_l6': 0.6483492122964524, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:12:05,045] Trial 248 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.11747548442542258, 'drop_l2': 0.12456071461312856, 'drop_l3': 0.2034617364489455, 'drop_l4': 0.18779603862676628, 'drop_l5': 0.47079743750811065, 'drop_l6': 0.6333145422684372, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:12:05,862] Trial 249 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.1008939812445934, 'drop_l2': 0.1593906078622068, 'drop_l3': 0.2070475868335413, 'drop_l4': 0.11415558340200763, 'drop_l5': 0.5559948903474609, 'drop_l6': 0.6300570783890491, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:12:05,897] Trial 250 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.11722939208837826, 'drop_l2': 0.10070160618296693, 'drop_l3': 0.21184390592913058, 'drop_l4': 0.18843344942029608, 'drop_l5': 0.5543364388298477, 'drop_l6': 0.6287189166427982, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:12:59,843] Trial 251 finished with value: 0.65625 and parameters: {'drop_l1': 0.12082940599439077, 'drop_l2': 0.1502545761304443, 'drop_l3': 0.14041384991743786, 'drop_l4': 0.13465514948986101, 'drop_l5': 0.5103056603399181, 'drop_l6': 0.6330179281061709, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:13:03,193] Trial 252 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.1164702478832661, 'drop_l2': 0.16201386252588867, 'drop_l3': 0.16259928641113736, 'drop_l4': 0.11016689135316679, 'drop_l5': 0.5522310916122364, 'drop_l6': 0.6681751722919389, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:13:04,192] Trial 254 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.11960676015194345, 'drop_l2': 0.143901375048767, 'drop_l3': 0.16543426290826566, 'drop_l4': 0.10087075622478925, 'drop_l5': 0.57479178470148, 'drop_l6': 0.6683487547081596, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:13:04,424] Trial 253 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.12177057543538977, 'drop_l2': 0.15197368620498397, 'drop_l3': 0.1623908200452576, 'drop_l4': 0.14290477799058435, 'drop_l5': 0.5753544501563453, 'drop_l6': 0.6632114071487719, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:13:57,029] Trial 255 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.11628338578778868, 'drop_l2': 0.15527896583262152, 'drop_l3': 0.11342163447859765, 'drop_l4': 0.10039430572809768, 'drop_l5': 0.5368219930289551, 'drop_l6': 0.6662656667987455, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:14:01,857] Trial 256 finished with value: 0.625 and parameters: {'drop_l1': 0.15399412446324678, 'drop_l2': 0.10000207861260188, 'drop_l3': 0.12170321191671644, 'drop_l4': 0.13471172761018113, 'drop_l5': 0.5029671234200328, 'drop_l6': 0.7293296364344082, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:14:02,803] Trial 258 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.3906868690004218, 'drop_l2': 0.10095185741646154, 'drop_l3': 0.11703837804185152, 'drop_l4': 0.12174879304028162, 'drop_l5': 0.47378892461964384, 'drop_l6': 0.6836612419149601, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:14:02,810] Trial 257 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.13960754362315314, 'drop_l2': 0.15240718401628572, 'drop_l3': 0.13208149814371398, 'drop_l4': 0.13726752662219807, 'drop_l5': 0.5045328150057662, 'drop_l6': 0.6572967840491499, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:14:51,348] Trial 259 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.13707184020287053, 'drop_l2': 0.14189414712658233, 'drop_l3': 0.13154843228091817, 'drop_l4': 0.13652802399112857, 'drop_l5': 0.5004696035228108, 'drop_l6': 0.6804294367784588, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:14:59,637] Trial 260 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.10073145065552049, 'drop_l2': 0.13669994305840788, 'drop_l3': 0.1412809527786723, 'drop_l4': 0.12494769501846666, 'drop_l5': 0.4789618805857087, 'drop_l6': 0.6833509210352384, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:15:00,261] Trial 261 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.1968185888776141, 'drop_l2': 0.11795965392877249, 'drop_l3': 0.14052112109952516, 'drop_l4': 0.1022675545377681, 'drop_l5': 0.4452384877085034, 'drop_l6': 0.6048653557244694, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:15:01,453] Trial 262 finished with value: 0.581250011920929 and parameters: {'drop_l1': 0.1879264825299757, 'drop_l2': 0.11717549394278011, 'drop_l3': 0.1884098321507801, 'drop_l4': 0.11651259521906583, 'drop_l5': 0.5815501025236676, 'drop_l6': 0.6061942771293617, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:15:41,804] Trial 263 finished with value: 0.668749988079071 and parameters: {'drop_l1': 0.10102647261219072, 'drop_l2': 0.11519837297724755, 'drop_l3': 0.18869490549833978, 'drop_l4': 0.20547165836830047, 'drop_l5': 0.5774307459011644, 'drop_l6': 0.6007048748655907, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:15:56,114] Trial 264 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.18048480050693572, 'drop_l2': 0.11673381660901432, 'drop_l3': 0.18255094754799475, 'drop_l4': 0.16858237986377303, 'drop_l5': 0.3295338400512796, 'drop_l6': 0.6057892533063898, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:15:56,730] Trial 266 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.1690250225759103, 'drop_l2': 0.11659262637826161, 'drop_l3': 0.1595261420213679, 'drop_l4': 0.10277743069089121, 'drop_l5': 0.4367545409637928, 'drop_l6': 0.630824837351015, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:15:58,682] Trial 265 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.18793970313794697, 'drop_l2': 0.1181649613763924, 'drop_l3': 0.1004251736833271, 'drop_l4': 0.10575452818413171, 'drop_l5': 0.42481469424173485, 'drop_l6': 0.6107118463322208, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:16:30,936] Trial 267 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.16009532987903355, 'drop_l2': 0.11677446394412741, 'drop_l3': 0.15600292415832723, 'drop_l4': 0.16502386498298374, 'drop_l5': 0.5731370428866528, 'drop_l6': 0.6208920279936533, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:16:52,171] Trial 268 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.16923728289386772, 'drop_l2': 0.12139505346431931, 'drop_l3': 0.18436178387685043, 'drop_l4': 0.11070576457032072, 'drop_l5': 0.4531606198695993, 'drop_l6': 0.6063395076431148, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:16:53,736] Trial 269 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.1785726100403952, 'drop_l2': 0.1275894168005472, 'drop_l3': 0.18201529977863842, 'drop_l4': 0.1654365310996549, 'drop_l5': 0.5765324106600219, 'drop_l6': 0.6064732155738307, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:16:53,792] Trial 270 finished with value: 0.6625000238418579 and parameters: {'drop_l1': 0.16774354373705655, 'drop_l2': 0.11181725609309517, 'drop_l3': 0.1568484509077714, 'drop_l4': 0.10343664223951722, 'drop_l5': 0.4166740378381588, 'drop_l6': 0.11978051800939044, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:17:18,280] Trial 271 finished with value: 0.625 and parameters: {'drop_l1': 0.7734483162895396, 'drop_l2': 0.10223531400085965, 'drop_l3': 0.18042562027477646, 'drop_l4': 0.17013891476814913, 'drop_l5': 0.5457805725793348, 'drop_l6': 0.5914979268332691, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:17:47,372] Trial 272 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.18102472246167367, 'drop_l2': 0.17723730599542736, 'drop_l3': 0.17287740658672915, 'drop_l4': 0.20227514198339758, 'drop_l5': 0.33478597076477057, 'drop_l6': 0.587267208633364, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:17:48,737] Trial 273 finished with value: 0.578125 and parameters: {'drop_l1': 0.10145742713197975, 'drop_l2': 0.17707608876842892, 'drop_l3': 0.1703842824096111, 'drop_l4': 0.2045181711562196, 'drop_l5': 0.334165714211388, 'drop_l6': 0.144656138304606, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:17:49,871] Trial 274 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.20089953998508983, 'drop_l2': 0.17128338730594245, 'drop_l3': 0.17001369865686963, 'drop_l4': 0.20486096890371988, 'drop_l5': 0.39180553695753184, 'drop_l6': 0.1607068954735198, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:18:04,791] Trial 275 finished with value: 0.640625 and parameters: {'drop_l1': 0.19748247685650433, 'drop_l2': 0.10018979690282129, 'drop_l3': 0.16496963711673415, 'drop_l4': 0.20148620816744303, 'drop_l5': 0.4115665412200615, 'drop_l6': 0.1345841047417659, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:18:41,649] Trial 276 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.19891096588689344, 'drop_l2': 0.1038502064091694, 'drop_l3': 0.15876448770448004, 'drop_l4': 0.1167205825517287, 'drop_l5': 0.43752995026688335, 'drop_l6': 0.6300204762509851, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:18:42,672] Trial 277 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.14801130446176383, 'drop_l2': 0.10059591546854332, 'drop_l3': 0.15658387341568042, 'drop_l4': 0.1490802714207884, 'drop_l5': 0.39341913907591314, 'drop_l6': 0.2551727666714085, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:18:43,832] Trial 278 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.1634547743143723, 'drop_l2': 0.13350248284340274, 'drop_l3': 0.15258787683653846, 'drop_l4': 0.12115107280556331, 'drop_l5': 0.43420576031162716, 'drop_l6': 0.2574649317039839, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:18:51,418] Trial 279 finished with value: 0.609375 and parameters: {'drop_l1': 0.10017893299264796, 'drop_l2': 0.13367070918428164, 'drop_l3': 0.1407751892923402, 'drop_l4': 0.10132360558741842, 'drop_l5': 0.3766734632187759, 'drop_l6': 0.642038542562743, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:19:26,648] Trial 281 finished with value: 0.640625 and parameters: {'drop_l1': 0.1663432495274891, 'drop_l2': 0.13140585690335216, 'drop_l3': 0.13872773335423333, 'drop_l4': 0.10491337303820937, 'drop_l5': 0.434631389922563, 'drop_l6': 0.6422594626366392, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:19:26,684] Trial 280 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.14532853143479457, 'drop_l2': 0.13295611322301343, 'drop_l3': 0.13773649480731456, 'drop_l4': 0.14965302838524575, 'drop_l5': 0.41272114357504136, 'drop_l6': 0.5942234705534329, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:19:28,663] Trial 282 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.16695385236133184, 'drop_l2': 0.1341796795578288, 'drop_l3': 0.14034228631071488, 'drop_l4': 0.10607994154302992, 'drop_l5': 0.41855852446231523, 'drop_l6': 0.6364852222940769, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:19:30,288] Trial 283 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.14153905940580225, 'drop_l2': 0.1523455972142361, 'drop_l3': 0.13534316758663462, 'drop_l4': 0.18157850298996644, 'drop_l5': 0.5421668575680973, 'drop_l6': 0.5900360011704125, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:20:09,494] Trial 285 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.17769662884782553, 'drop_l2': 0.15377335163634248, 'drop_l3': 0.3654133601591911, 'drop_l4': 0.12841470252034987, 'drop_l5': 0.4597085591224994, 'drop_l6': 0.6173545077368088, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:20:09,860] Trial 284 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.13822164835207446, 'drop_l2': 0.14905676001259616, 'drop_l3': 0.3708847917407315, 'drop_l4': 0.13192722114777433, 'drop_l5': 0.4501808798972998, 'drop_l6': 0.6177639516594842, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:20:11,862] Trial 286 finished with value: 0.668749988079071 and parameters: {'drop_l1': 0.10010707210899947, 'drop_l2': 0.1527807594884556, 'drop_l3': 0.10798407380480869, 'drop_l4': 0.17774374758447747, 'drop_l5': 0.5943919840151829, 'drop_l6': 0.7020696440944239, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:20:12,344] Trial 287 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.1784295302255111, 'drop_l2': 0.11219477465317125, 'drop_l3': 0.15557235054091798, 'drop_l4': 0.12730545417893022, 'drop_l5': 0.47096277543198095, 'drop_l6': 0.6218934050309111, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:20:51,774] Trial 288 finished with value: 0.640625 and parameters: {'drop_l1': 0.10020267783240344, 'drop_l2': 0.11695752356071605, 'drop_l3': 0.1258843808675871, 'drop_l4': 0.10311061563208933, 'drop_l5': 0.3709492952504314, 'drop_l6': 0.6244325002573868, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:20:53,661] Trial 291 finished with value: 0.574999988079071 and parameters: {'drop_l1': 0.11398793309246362, 'drop_l2': 0.18720902828680658, 'drop_l3': 0.10807309463343168, 'drop_l4': 0.10025188476647297, 'drop_l5': 0.44833298252559733, 'drop_l6': 0.7028557140201286, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:20:53,687] Trial 290 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.12618614075425347, 'drop_l2': 0.1601815243898862, 'drop_l3': 0.10092064859274477, 'drop_l4': 0.1580853840085292, 'drop_l5': 0.6235285049695307, 'drop_l6': 0.6517665261505707, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:20:53,843] Trial 289 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.1302796104230894, 'drop_l2': 0.1628106730595857, 'drop_l3': 0.38174623828041926, 'drop_l4': 0.2199803701912357, 'drop_l5': 0.35437738359500404, 'drop_l6': 0.6217763576266212, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:21:34,559] Trial 292 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.11924662483839295, 'drop_l2': 0.18480441774913337, 'drop_l3': 0.11472417290087783, 'drop_l4': 0.16046725405577683, 'drop_l5': 0.5982385895004272, 'drop_l6': 0.6923410194224489, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:21:37,590] Trial 295 finished with value: 0.625 and parameters: {'drop_l1': 0.11446896194656, 'drop_l2': 0.14794331589921936, 'drop_l3': 0.37006896754665, 'drop_l4': 0.15121214497805052, 'drop_l5': 0.5986104280092561, 'drop_l6': 0.5933246331659358, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:21:37,922] Trial 293 finished with value: 0.59375 and parameters: {'drop_l1': 0.13027307733569662, 'drop_l2': 0.1624030124413491, 'drop_l3': 0.10073296484568069, 'drop_l4': 0.1427699469046158, 'drop_l5': 0.6232582260311401, 'drop_l6': 0.6554905965732801, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:21:38,039] Trial 294 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.12596203365062575, 'drop_l2': 0.15680587893772657, 'drop_l3': 0.11160034814766685, 'drop_l4': 0.1426930429089252, 'drop_l5': 0.6056126968361804, 'drop_l6': 0.5834171497652489, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:22:16,785] Trial 296 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.10001325466531229, 'drop_l2': 0.16461382851219133, 'drop_l3': 0.8166328794197568, 'drop_l4': 0.1369943388047214, 'drop_l5': 0.6178749090973281, 'drop_l6': 0.20661163966918972, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:22:21,782] Trial 297 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.13180676964906993, 'drop_l2': 0.16189135048174552, 'drop_l3': 0.3940408950026094, 'drop_l4': 0.1364191906847676, 'drop_l5': 0.6214479771010155, 'drop_l6': 0.6495338132826738, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:22:21,795] Trial 298 finished with value: 0.65625 and parameters: {'drop_l1': 0.10025757644228467, 'drop_l2': 0.13893671249795994, 'drop_l3': 0.20065871857135636, 'drop_l4': 0.18191767636662654, 'drop_l5': 0.5862286433773977, 'drop_l6': 0.5831112732547259, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:22:21,904] Trial 299 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.10063266404387711, 'drop_l2': 0.14247587414947874, 'drop_l3': 0.34255341156690144, 'drop_l4': 0.12218073659969952, 'drop_l5': 0.5878021656041115, 'drop_l6': 0.600966581418754, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:22:50,558] Trial 300 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.13147287094136917, 'drop_l2': 0.139662004247827, 'drop_l3': 0.19650749798918354, 'drop_l4': 0.18608548121136836, 'drop_l5': 0.5863164498507968, 'drop_l6': 0.6400825999731005, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:23:02,392] Trial 301 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.10066330052333398, 'drop_l2': 0.14189590964920723, 'drop_l3': 0.2221275978817855, 'drop_l4': 0.17983098076650791, 'drop_l5': 0.5926960080993481, 'drop_l6': 0.7043259467984916, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:23:03,308] Trial 302 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.10063619838376157, 'drop_l2': 0.13994870923844427, 'drop_l3': 0.20445143557331122, 'drop_l4': 0.16779934790790418, 'drop_l5': 0.5898024498111437, 'drop_l6': 0.7499942684557882, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:23:03,367] Trial 303 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.10307125764996086, 'drop_l2': 0.12693263531532395, 'drop_l3': 0.21804044122856325, 'drop_l4': 0.1747270794756844, 'drop_l5': 0.49161623759560075, 'drop_l6': 0.5732561520209972, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:23:20,462] Trial 304 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.10028122741323664, 'drop_l2': 0.11323772466528242, 'drop_l3': 0.22400914188411256, 'drop_l4': 0.17841528038963803, 'drop_l5': 0.48103028560830435, 'drop_l6': 0.6679926079798404, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:23:42,015] Trial 305 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.1006478202261392, 'drop_l2': 0.12336097847206837, 'drop_l3': 0.21444052394076274, 'drop_l4': 0.1711533679757975, 'drop_l5': 0.48746358512233295, 'drop_l6': 0.741157731208408, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:23:43,592] Trial 307 finished with value: 0.675000011920929 and parameters: {'drop_l1': 0.11344370445145578, 'drop_l2': 0.11393165034850398, 'drop_l3': 0.2038853887961708, 'drop_l4': 0.21363711633130736, 'drop_l5': 0.5711741977252519, 'drop_l6': 0.7422840649754265, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:23:44,600] Trial 306 finished with value: 0.65625 and parameters: {'drop_l1': 0.10068389818668881, 'drop_l2': 0.12476081090132364, 'drop_l3': 0.1936680756034252, 'drop_l4': 0.16409139719966023, 'drop_l5': 0.5681853183087615, 'drop_l6': 0.7812928747211563, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:23:52,959] Trial 308 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.1158673506067522, 'drop_l2': 0.18962954625124315, 'drop_l3': 0.20067490781026198, 'drop_l4': 0.21835365969604265, 'drop_l5': 0.5754624925256087, 'drop_l6': 0.8488788612674516, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:24:23,008] Trial 309 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.11465915819044899, 'drop_l2': 0.1950319943405286, 'drop_l3': 0.20107656299372836, 'drop_l4': 0.20285095284202284, 'drop_l5': 0.5695883337245973, 'drop_l6': 0.8362624163820862, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:24:23,503] Trial 310 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.11554459560957793, 'drop_l2': 0.19711311064353154, 'drop_l3': 0.20364411824212922, 'drop_l4': 0.21244366884082114, 'drop_l5': 0.5695600609378971, 'drop_l6': 0.7698349961483764, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:24:23,517] Trial 311 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.14451550842187014, 'drop_l2': 0.10090242162506496, 'drop_l3': 0.18680716598681968, 'drop_l4': 0.21533534797653642, 'drop_l5': 0.5654552845466906, 'drop_l6': 0.7804532170336754, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:24:29,010] Trial 312 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.1490598368755617, 'drop_l2': 0.10070675803633235, 'drop_l3': 0.1941745961551534, 'drop_l4': 0.20430188235407257, 'drop_l5': 0.5521943636954574, 'drop_l6': 0.7816198032436633, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:25:05,136] Trial 314 finished with value: 0.625 and parameters: {'drop_l1': 0.10195148000244936, 'drop_l2': 0.10035509437709264, 'drop_l3': 0.23303640237310289, 'drop_l4': 0.22689347539835594, 'drop_l5': 0.5545000574929936, 'drop_l6': 0.7889404673290581, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:25:05,161] Trial 315 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.1449553378827594, 'drop_l2': 0.10307020908367705, 'drop_l3': 0.23082091975202773, 'drop_l4': 0.2277582524077516, 'drop_l5': 0.56141407305186, 'drop_l6': 0.8309230048891094, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:25:05,164] Trial 313 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.14410619064193783, 'drop_l2': 0.10157561464614705, 'drop_l3': 0.17025760147656657, 'drop_l4': 0.21856544199344755, 'drop_l5': 0.5600826357761792, 'drop_l6': 0.7617294852921775, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:25:06,808] Trial 316 finished with value: 0.640625 and parameters: {'drop_l1': 0.14372654294603043, 'drop_l2': 0.11005029713455416, 'drop_l3': 0.19323336978300532, 'drop_l4': 0.22543828759024195, 'drop_l5': 0.5613225637861332, 'drop_l6': 0.7763411135064545, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:25:47,501] Trial 319 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.1177289181449258, 'drop_l2': 0.1758074740246435, 'drop_l3': 0.18716622429499963, 'drop_l4': 0.2083546688616201, 'drop_l5': 0.5130524528515865, 'drop_l6': 0.7590656188971439, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:25:48,343] Trial 320 finished with value: 0.625 and parameters: {'drop_l1': 0.12262481476881051, 'drop_l2': 0.12631625310035366, 'drop_l3': 0.21498644242158346, 'drop_l4': 0.20120213438308404, 'drop_l5': 0.5790063224173583, 'drop_l6': 0.7889827774092959, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:25:49,234] Trial 318 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.11868601270628996, 'drop_l2': 0.17475506491353307, 'drop_l3': 0.1857382196068325, 'drop_l4': 0.19845313331384287, 'drop_l5': 0.401626371041018, 'drop_l6': 0.8313619874011099, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:25:49,251] Trial 317 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.141967698816553, 'drop_l2': 0.18143462703183033, 'drop_l3': 0.18701841177327733, 'drop_l4': 0.21190950146864343, 'drop_l5': 0.5107110583682769, 'drop_l6': 0.8398875934610489, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:26:30,736] Trial 321 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.15824876907809424, 'drop_l2': 0.12608092676152732, 'drop_l3': 0.1791545522443787, 'drop_l4': 0.1902392161020556, 'drop_l5': 0.533044171635515, 'drop_l6': 0.7950190168738337, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:26:32,415] Trial 324 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.10155668562804543, 'drop_l2': 0.14706950277864253, 'drop_l3': 0.21189056883922447, 'drop_l4': 0.1842819299241475, 'drop_l5': 0.5976116633945531, 'drop_l6': 0.8654690689137214, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:26:32,694] Trial 323 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.13121367736712217, 'drop_l2': 0.14646159693850552, 'drop_l3': 0.20593462886843433, 'drop_l4': 0.1897605006626029, 'drop_l5': 0.5886545821513565, 'drop_l6': 0.7462780091428441, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:26:32,771] Trial 322 finished with value: 0.5531250238418579 and parameters: {'drop_l1': 0.1005316412008522, 'drop_l2': 0.13176670163914994, 'drop_l3': 0.2060432685204931, 'drop_l4': 0.18878497853669854, 'drop_l5': 0.6009450809316441, 'drop_l6': 0.7538235732447464, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:27:13,250] Trial 325 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.10218700332696418, 'drop_l2': 0.13554099359060018, 'drop_l3': 0.21254330256087758, 'drop_l4': 0.15654409238923075, 'drop_l5': 0.5996355850248124, 'drop_l6': 0.7432633916507688, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:27:14,075] Trial 327 finished with value: 0.640625 and parameters: {'drop_l1': 0.11128389027253474, 'drop_l2': 0.14071916595922862, 'drop_l3': 0.23719811431241009, 'drop_l4': 0.16002751405904872, 'drop_l5': 0.6069151015210135, 'drop_l6': 0.8150861067349309, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:27:14,748] Trial 328 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.10003499012381621, 'drop_l2': 0.1423253215773993, 'drop_l3': 0.16397575989267865, 'drop_l4': 0.15811352410787793, 'drop_l5': 0.5784092180490691, 'drop_l6': 0.7230792014370295, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:27:16,345] Trial 326 finished with value: 0.5687500238418579 and parameters: {'drop_l1': 0.13345116480590147, 'drop_l2': 0.12885766298190932, 'drop_l3': 0.1544590044316989, 'drop_l4': 0.15779951224468824, 'drop_l5': 0.6401014778047589, 'drop_l6': 0.7493161286674519, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:27:54,930] Trial 329 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.13794713296272085, 'drop_l2': 0.11273176699248523, 'drop_l3': 0.15248322583722201, 'drop_l4': 0.2442341599108339, 'drop_l5': 0.5837766024935636, 'drop_l6': 0.7270505350930551, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:27:56,526] Trial 330 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.15712816595785345, 'drop_l2': 0.11470977313363216, 'drop_l3': 0.25034732053819986, 'drop_l4': 0.1664637016377271, 'drop_l5': 0.38802852943230925, 'drop_l6': 0.5544028906923452, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:27:56,855] Trial 332 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.15380564631681148, 'drop_l2': 0.1134076397950804, 'drop_l3': 0.1976726202047761, 'drop_l4': 0.23852704030218602, 'drop_l5': 0.5686080534182512, 'drop_l6': 0.8128454842543865, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:27:57,251] Trial 331 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.1001684314805868, 'drop_l2': 0.11222578413638498, 'drop_l3': 0.1512638644751057, 'drop_l4': 0.18313544770402065, 'drop_l5': 0.5696241873900775, 'drop_l6': 0.8100733965800535, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:28:34,839] Trial 333 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.15853051857409733, 'drop_l2': 0.15665549913194993, 'drop_l3': 0.1955828048455189, 'drop_l4': 0.17283273571937832, 'drop_l5': 0.3250610917534627, 'drop_l6': 0.77205890013137, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:28:38,591] Trial 335 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.11835699006101053, 'drop_l2': 0.16371511346926798, 'drop_l3': 0.17909159991786575, 'drop_l4': 0.13977097438291805, 'drop_l5': 0.5361179354803436, 'drop_l6': 0.8601388217149828, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:28:38,628] Trial 336 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.11930661061215349, 'drop_l2': 0.15992401863080377, 'drop_l3': 0.17740771562058033, 'drop_l4': 0.13661803299416006, 'drop_l5': 0.4196788070019434, 'drop_l6': 0.8863736695164379, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:28:39,077] Trial 334 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.1000953367407301, 'drop_l2': 0.15594585491601404, 'drop_l3': 0.16786856767095498, 'drop_l4': 0.11978448267962463, 'drop_l5': 0.5681052074141872, 'drop_l6': 0.7663847004895515, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:29:13,744] Trial 337 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.12039990013432257, 'drop_l2': 0.170164736528424, 'drop_l3': 0.12288996716132708, 'drop_l4': 0.20629743807374723, 'drop_l5': 0.5367618296875438, 'drop_l6': 0.5644234501020481, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:29:19,945] Trial 340 finished with value: 0.625 and parameters: {'drop_l1': 0.1818796157603144, 'drop_l2': 0.11933397059457862, 'drop_l3': 0.222548827347563, 'drop_l4': 0.20454261495638135, 'drop_l5': 0.543498353661167, 'drop_l6': 0.5728901705357161, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:29:21,069] Trial 339 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.13221343229467616, 'drop_l2': 0.3916574198747207, 'drop_l3': 0.19599571143625832, 'drop_l4': 0.20954054507225417, 'drop_l5': 0.5865373244007441, 'drop_l6': 0.5801574406779213, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:29:21,071] Trial 338 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.12743612188094874, 'drop_l2': 0.15005556216045637, 'drop_l3': 0.17036203605432398, 'drop_l4': 0.20789896672740452, 'drop_l5': 0.5806559896817525, 'drop_l6': 0.5867995392266405, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:29:46,547] Trial 341 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.1323126992714102, 'drop_l2': 0.39348338085041146, 'drop_l3': 0.2204164058684073, 'drop_l4': 0.2248346921013048, 'drop_l5': 0.4561053054241693, 'drop_l6': 0.5868390360291398, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:30:01,540] Trial 342 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.21040204677136717, 'drop_l2': 0.10000906891734289, 'drop_l3': 0.1955668081730571, 'drop_l4': 0.17631432053540794, 'drop_l5': 0.44769420954288996, 'drop_l6': 0.5837538207833909, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:30:02,965] Trial 344 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.11380689875329451, 'drop_l2': 0.2111495138579687, 'drop_l3': 0.2015114419537755, 'drop_l4': 0.22839558847625707, 'drop_l5': 0.45482110563048633, 'drop_l6': 0.7335278718761868, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:30:02,989] Trial 343 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.11239911177234697, 'drop_l2': 0.20649245222500845, 'drop_l3': 0.20486802284635036, 'drop_l4': 0.14921892754098356, 'drop_l5': 0.44587673589144583, 'drop_l6': 0.7347640159903929, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:30:18,908] Trial 345 finished with value: 0.625 and parameters: {'drop_l1': 0.11345648848930426, 'drop_l2': 0.14482648992840857, 'drop_l3': 0.20338997107772513, 'drop_l4': 0.15298969173946816, 'drop_l5': 0.5625877659758893, 'drop_l6': 0.7068546964887149, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:30:42,019] Trial 346 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.11600791268254484, 'drop_l2': 0.1902130862878388, 'drop_l3': 0.12173243796050091, 'drop_l4': 0.25665101431302123, 'drop_l5': 0.3513117275772455, 'drop_l6': 0.5545208618202098, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:30:45,000] Trial 347 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1708382548962289, 'drop_l2': 0.13507323360692228, 'drop_l3': 0.13486780163861511, 'drop_l4': 0.11923259474515217, 'drop_l5': 0.556851827611224, 'drop_l6': 0.6147722986130266, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:30:45,585] Trial 348 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.1427490743583882, 'drop_l2': 0.1005683403051836, 'drop_l3': 0.12953273449773833, 'drop_l4': 0.11848061585398174, 'drop_l5': 0.5513023258605936, 'drop_l6': 0.6146957856623657, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:30:53,862] Trial 349 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.14967434545129782, 'drop_l2': 0.17568473424992154, 'drop_l3': 0.11678269526598033, 'drop_l4': 0.21905231795017155, 'drop_l5': 0.5478264314624274, 'drop_l6': 0.552026163417865, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:31:21,318] Trial 350 finished with value: 0.640625 and parameters: {'drop_l1': 0.16954092077550875, 'drop_l2': 0.16943622589556917, 'drop_l3': 0.1789924277895181, 'drop_l4': 0.21305707233667087, 'drop_l5': 0.3636684643245809, 'drop_l6': 0.5627900265941489, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:31:26,583] Trial 352 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.15621015933533378, 'drop_l2': 0.17430194961888845, 'drop_l3': 0.10237785073666641, 'drop_l4': 0.201120339504732, 'drop_l5': 0.5251809139821675, 'drop_l6': 0.5615562394659072, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:31:26,647] Trial 351 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.15926822431659765, 'drop_l2': 0.8960223181361959, 'drop_l3': 0.16855871801391165, 'drop_l4': 0.19278619050332604, 'drop_l5': 0.3752557460235208, 'drop_l6': 0.6036226671643442, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:31:30,746] Trial 353 finished with value: 0.59375 and parameters: {'drop_l1': 0.1280173253485709, 'drop_l2': 0.12604953171678224, 'drop_l3': 0.2642501414017725, 'drop_l4': 0.19427577393590184, 'drop_l5': 0.5256963753874959, 'drop_l6': 0.5413389408352445, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:31:52,949] Trial 354 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.15511757979754393, 'drop_l2': 0.122867017577317, 'drop_l3': 0.2550041284348998, 'drop_l4': 0.18867842920581912, 'drop_l5': 0.404479205915365, 'drop_l6': 0.6167978162907226, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:32:07,393] Trial 355 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.13120579712576871, 'drop_l2': 0.12702771184864028, 'drop_l3': 0.1660477968737441, 'drop_l4': 0.17123412296806956, 'drop_l5': 0.2572549221072805, 'drop_l6': 0.6005888978601367, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:32:09,643] Trial 357 finished with value: 0.640625 and parameters: {'drop_l1': 0.19282672478131713, 'drop_l2': 0.12301106435976006, 'drop_l3': 0.15999411320332396, 'drop_l4': 0.14369214032549918, 'drop_l5': 0.4020314134758403, 'drop_l6': 0.5961426733357059, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:32:10,086] Trial 356 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.13509305033838637, 'drop_l2': 0.12564619228705132, 'drop_l3': 0.18378945756085496, 'drop_l4': 0.1678563019929814, 'drop_l5': 0.5902507744893836, 'drop_l6': 0.5928984267586307, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:32:25,284] Trial 358 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.11409097910201257, 'drop_l2': 0.19109120638549434, 'drop_l3': 0.15182778098878194, 'drop_l4': 0.23252076700981772, 'drop_l5': 0.5787731574438834, 'drop_l6': 0.6808471350138678, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:32:48,673] Trial 359 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.13942761128741948, 'drop_l2': 0.10041471618998421, 'drop_l3': 0.1582321727004648, 'drop_l4': 0.16033178345875015, 'drop_l5': 0.23569519586765353, 'drop_l6': 0.600271196400281, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:32:50,054] Trial 361 finished with value: 0.668749988079071 and parameters: {'drop_l1': 0.12923801053990314, 'drop_l2': 0.1021290021476394, 'drop_l3': 0.14654900362155882, 'drop_l4': 0.2399250397926851, 'drop_l5': 0.23373745382802585, 'drop_l6': 0.5719811409600642, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:32:50,726] Trial 360 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.10066880836417653, 'drop_l2': 0.1002558649161096, 'drop_l3': 0.1715737247665575, 'drop_l4': 0.10027947959940606, 'drop_l5': 0.25540179932936613, 'drop_l6': 0.5947424206505316, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:32:59,580] Trial 362 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.13658436862655338, 'drop_l2': 0.10268744780956636, 'drop_l3': 0.35389607152575664, 'drop_l4': 0.16292743663685866, 'drop_l5': 0.2590193129823647, 'drop_l6': 0.5940743216677854, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:33:30,342] Trial 363 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.10083653823108228, 'drop_l2': 0.10129134580192131, 'drop_l3': 0.24225248499583577, 'drop_l4': 0.1828588637123332, 'drop_l5': 0.2516146432165715, 'drop_l6': 0.6373810420228903, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:33:33,348] Trial 365 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.1308902482986928, 'drop_l2': 0.10161807805946871, 'drop_l3': 0.14423994823927175, 'drop_l4': 0.2483677573179754, 'drop_l5': 0.2732854134796655, 'drop_l6': 0.5822134049080819, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:33:33,363] Trial 364 finished with value: 0.6781250238418579 and parameters: {'drop_l1': 0.10017007151309261, 'drop_l2': 0.10047365498173329, 'drop_l3': 0.12320673097946624, 'drop_l4': 0.24901222105995433, 'drop_l5': 0.2629748344146769, 'drop_l6': 0.5752720987728521, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:33:36,444] Trial 366 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.10275839012426288, 'drop_l2': 0.10158096417976722, 'drop_l3': 0.23194826072533287, 'drop_l4': 0.26822251072575476, 'drop_l5': 0.24560737979257816, 'drop_l6': 0.5766578644219664, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:34:09,601] Trial 367 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.10410064361724704, 'drop_l2': 0.10050834755411213, 'drop_l3': 0.14408730309820733, 'drop_l4': 0.18161275316389733, 'drop_l5': 0.25304649934724105, 'drop_l6': 0.6316773205883799, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:34:15,043] Trial 368 finished with value: 0.609375 and parameters: {'drop_l1': 0.10384289970661806, 'drop_l2': 0.1000987488357817, 'drop_l3': 0.1440754905839521, 'drop_l4': 0.2729121555840689, 'drop_l5': 0.25173405731060355, 'drop_l6': 0.570602168649843, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:34:15,517] Trial 370 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.10009957139019658, 'drop_l2': 0.4129304377755284, 'drop_l3': 0.13965358878911405, 'drop_l4': 0.23770434731207343, 'drop_l5': 0.25248641345862344, 'drop_l6': 0.5684167380467806, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:34:17,019] Trial 369 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.10043765435799545, 'drop_l2': 0.5117718699981846, 'drop_l3': 0.13188639073108727, 'drop_l4': 0.2687261649483248, 'drop_l5': 0.25823277822930124, 'drop_l6': 0.5666822576320019, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:34:42,455] Trial 371 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.1004532681462184, 'drop_l2': 0.10149201449311258, 'drop_l3': 0.12512769929368717, 'drop_l4': 0.2638371236913506, 'drop_l5': 0.2539215594927664, 'drop_l6': 0.5688450605710331, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:34:57,942] Trial 374 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.10042967556141809, 'drop_l2': 0.11195135306429667, 'drop_l3': 0.11535837669428645, 'drop_l4': 0.29028772959840454, 'drop_l5': 0.2162055111124198, 'drop_l6': 0.5392994929599101, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:34:58,500] Trial 372 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.10324431456539118, 'drop_l2': 0.11156661360631139, 'drop_l3': 0.10301996812117005, 'drop_l4': 0.256069587278244, 'drop_l5': 0.2378179575136962, 'drop_l6': 0.5451228707926171, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:34:58,520] Trial 373 finished with value: 0.59375 and parameters: {'drop_l1': 0.10037439406195289, 'drop_l2': 0.11540324707859963, 'drop_l3': 0.1251283823893203, 'drop_l4': 0.23205397884622342, 'drop_l5': 0.2901903458150247, 'drop_l6': 0.5418718895086776, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:35:14,873] Trial 375 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.12229246168749248, 'drop_l2': 0.11464347754467602, 'drop_l3': 0.24152015018380388, 'drop_l4': 0.2891794948697783, 'drop_l5': 0.290163286414331, 'drop_l6': 0.5431058427547155, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:35:39,139] Trial 376 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.12380658637538562, 'drop_l2': 0.11462217045961638, 'drop_l3': 0.10051315949044891, 'drop_l4': 0.23647865474915936, 'drop_l5': 0.21892740469018468, 'drop_l6': 0.6028488592719503, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:35:39,147] Trial 377 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.12404186915713199, 'drop_l2': 0.11370533562666697, 'drop_l3': 0.265964047894635, 'drop_l4': 0.22308820131217694, 'drop_l5': 0.26839365994547426, 'drop_l6': 0.6057455734095409, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:35:40,589] Trial 378 finished with value: 0.640625 and parameters: {'drop_l1': 0.12033461937182496, 'drop_l2': 0.1007923054804433, 'drop_l3': 0.24950483273703494, 'drop_l4': 0.23555338024783137, 'drop_l5': 0.22816752675414018, 'drop_l6': 0.6010921276964247, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:35:50,152] Trial 379 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.12704708630850298, 'drop_l2': 0.13190283352324927, 'drop_l3': 0.16225561966457236, 'drop_l4': 0.2504137505963841, 'drop_l5': 0.21201315270659798, 'drop_l6': 0.6010607296712189, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:36:22,054] Trial 381 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.14344248636600618, 'drop_l2': 0.1302376110894242, 'drop_l3': 0.15878903144760942, 'drop_l4': 0.22310830468265724, 'drop_l5': 0.2756207589710779, 'drop_l6': 0.628321993647014, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:36:22,587] Trial 382 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.14512226064033418, 'drop_l2': 0.13280250095541338, 'drop_l3': 0.16483225607814406, 'drop_l4': 0.2207146563594844, 'drop_l5': 0.2713589788124195, 'drop_l6': 0.585604525103711, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:36:22,609] Trial 380 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.14455077407632166, 'drop_l2': 0.5376496266752737, 'drop_l3': 0.6579877190461259, 'drop_l4': 0.24050553375762584, 'drop_l5': 0.2026388842737018, 'drop_l6': 0.5853388134819735, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:36:26,309] Trial 383 finished with value: 0.609375 and parameters: {'drop_l1': 0.14568258665008116, 'drop_l2': 0.13016978376734206, 'drop_l3': 0.18026035946391877, 'drop_l4': 0.2158395379856073, 'drop_l5': 0.27074124629052865, 'drop_l6': 0.582590119690258, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:37:04,794] Trial 385 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.11760318765266572, 'drop_l2': 0.12792374160200878, 'drop_l3': 0.18181367469366339, 'drop_l4': 0.1815609197099513, 'drop_l5': 0.29902574557286654, 'drop_l6': 0.621437992837162, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:37:05,226] Trial 386 finished with value: 0.640625 and parameters: {'drop_l1': 0.1139732883091771, 'drop_l2': 0.12179070498672455, 'drop_l3': 0.18245019725754827, 'drop_l4': 0.18219788728847375, 'drop_l5': 0.23817946383474817, 'drop_l6': 0.6341810040677128, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:37:05,257] Trial 384 finished with value: 0.609375 and parameters: {'drop_l1': 0.1418655551580417, 'drop_l2': 0.10140517455725227, 'drop_l3': 0.17994109768299255, 'drop_l4': 0.18297710758967795, 'drop_l5': 0.23110268315015292, 'drop_l6': 0.5836365079026083, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:37:05,674] Trial 387 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.11512226202751563, 'drop_l2': 0.11894602850329916, 'drop_l3': 0.226161712870587, 'drop_l4': 0.1808758422716359, 'drop_l5': 0.30517913880700326, 'drop_l6': 0.6302819286716994, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:37:48,527] Trial 391 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.10125208700998976, 'drop_l2': 0.13895751715044072, 'drop_l3': 0.14700233793454806, 'drop_l4': 0.20064740421280944, 'drop_l5': 0.24929466608303055, 'drop_l6': 0.5551717835530525, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:37:48,594] Trial 390 finished with value: 0.609375 and parameters: {'drop_l1': 0.10178877510580094, 'drop_l2': 0.10066904217594828, 'drop_l3': 0.22422512863806343, 'drop_l4': 0.20014372644462372, 'drop_l5': 0.24977684178919893, 'drop_l6': 0.5605443625985976, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:37:49,075] Trial 389 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.16161719637885538, 'drop_l2': 0.10241521947691945, 'drop_l3': 0.2187231122430893, 'drop_l4': 0.2041333104036942, 'drop_l5': 0.24769622712961242, 'drop_l6': 0.5602907291670816, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:37:49,152] Trial 388 finished with value: 0.609375 and parameters: {'drop_l1': 0.11198750421024388, 'drop_l2': 0.1008633612515805, 'drop_l3': 0.1437607142440867, 'drop_l4': 0.1963895622236493, 'drop_l5': 0.2250306935953523, 'drop_l6': 0.5573363996720332, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:38:30,459] Trial 395 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.13466639200501682, 'drop_l2': 0.14375453845387484, 'drop_l3': 0.15512580326882122, 'drop_l4': 0.25354399693200114, 'drop_l5': 0.2824342563074707, 'drop_l6': 0.6121749229513299, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:38:30,726] Trial 392 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.16522830561242025, 'drop_l2': 0.146504782156241, 'drop_l3': 0.15199435985221801, 'drop_l4': 0.14774329143838055, 'drop_l5': 0.2262702308828354, 'drop_l6': 0.6576189078568655, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:38:30,878] Trial 393 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1614886080671228, 'drop_l2': 0.10035141061387176, 'drop_l3': 0.1496818080650533, 'drop_l4': 0.1443119255029267, 'drop_l5': 0.22745790530625337, 'drop_l6': 0.6152768977024157, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:38:30,892] Trial 394 finished with value: 0.65625 and parameters: {'drop_l1': 0.12840122750468724, 'drop_l2': 0.14852664011428637, 'drop_l3': 0.1675064547927583, 'drop_l4': 0.24661154402186347, 'drop_l5': 0.280840863755143, 'drop_l6': 0.6571068744624285, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:39:10,167] Trial 397 finished with value: 0.578125 and parameters: {'drop_l1': 0.10004799382492395, 'drop_l2': 0.11674492568741215, 'drop_l3': 0.1717426055812547, 'drop_l4': 0.1677384684101112, 'drop_l5': 0.25979632170602684, 'drop_l6': 0.5955706220932118, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:39:12,768] Trial 399 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.12925529192633228, 'drop_l2': 0.33646931436038063, 'drop_l3': 0.12520477881824618, 'drop_l4': 0.24230946615041574, 'drop_l5': 0.31390536238132194, 'drop_l6': 0.6491557297608087, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:39:12,802] Trial 396 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.12695357890701592, 'drop_l2': 0.10069125102872353, 'drop_l3': 0.16739191320102803, 'drop_l4': 0.1539724819022276, 'drop_l5': 0.23486911674971703, 'drop_l6': 0.7872890213010045, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:39:12,928] Trial 398 finished with value: 0.640625 and parameters: {'drop_l1': 0.1275162471548224, 'drop_l2': 0.14495778806526674, 'drop_l3': 0.17105576897775454, 'drop_l4': 0.2486555708941918, 'drop_l5': 0.281481076163453, 'drop_l6': 0.6501014607173345, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:39:49,643] Trial 400 finished with value: 0.59375 and parameters: {'drop_l1': 0.12983159660621368, 'drop_l2': 0.15482716723953868, 'drop_l3': 0.12632762165544958, 'drop_l4': 0.24367812274000883, 'drop_l5': 0.28646121520795476, 'drop_l6': 0.5793674774220592, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:39:53,741] Trial 403 finished with value: 0.625 and parameters: {'drop_l1': 0.17491245291191795, 'drop_l2': 0.16125866782932707, 'drop_l3': 0.5162246103420138, 'drop_l4': 0.22548987816468763, 'drop_l5': 0.2654930495496177, 'drop_l6': 0.5756657897708968, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:39:53,762] Trial 402 finished with value: 0.625 and parameters: {'drop_l1': 0.11474996848868345, 'drop_l2': 0.15621032529591466, 'drop_l3': 0.2659018474087369, 'drop_l4': 0.2585144944538029, 'drop_l5': 0.2833514662384911, 'drop_l6': 0.5789528802853787, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:39:53,766] Trial 401 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.13079165125486913, 'drop_l2': 0.15552170143638222, 'drop_l3': 0.1928956606577206, 'drop_l4': 0.2610427746884894, 'drop_l5': 0.26481323180901334, 'drop_l6': 0.6462591877097502, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:40:28,776] Trial 404 finished with value: 0.5562499761581421 and parameters: {'drop_l1': 0.7368130721220243, 'drop_l2': 0.12393485208917819, 'drop_l3': 0.1910023441502401, 'drop_l4': 0.17419958032509264, 'drop_l5': 0.2082205082158327, 'drop_l6': 0.5740479656389958, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:40:34,656] Trial 405 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.1016012881481973, 'drop_l2': 0.4345159719697769, 'drop_l3': 0.19056262623748849, 'drop_l4': 0.32205768506314086, 'drop_l5': 0.4272151665663798, 'drop_l6': 0.6609967947476237, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:40:35,165] Trial 406 finished with value: 0.640625 and parameters: {'drop_l1': 0.15651915143478165, 'drop_l2': 0.8175162998707566, 'drop_l3': 0.18260016728878745, 'drop_l4': 0.21638125985077916, 'drop_l5': 0.6113467215554815, 'drop_l6': 0.6295719438306857, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:40:35,408] Trial 407 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.15056559138899026, 'drop_l2': 0.1260307349838267, 'drop_l3': 0.18941497688913628, 'drop_l4': 0.32689047556374307, 'drop_l5': 0.3855876718957301, 'drop_l6': 0.6641018231860939, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:41:05,115] Trial 408 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.10162960418170816, 'drop_l2': 0.1274167521503845, 'drop_l3': 0.2394873877025825, 'drop_l4': 0.1271134135247392, 'drop_l5': 0.24519108916903345, 'drop_l6': 0.6646502568658617, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:41:16,787] Trial 409 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.11345505148866285, 'drop_l2': 0.1265192719586382, 'drop_l3': 0.2999453518606111, 'drop_l4': 0.1322078297473353, 'drop_l5': 0.2633654589171572, 'drop_l6': 0.6343664778683248, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:41:18,589] Trial 411 finished with value: 0.5874999761581421 and parameters: {'drop_l1': 0.1016539881208468, 'drop_l2': 0.11519167398115092, 'drop_l3': 0.24308197116950753, 'drop_l4': 0.2066970988761892, 'drop_l5': 0.2602722580739943, 'drop_l6': 0.5286488692436062, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:41:19,036] Trial 410 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.14726568745948637, 'drop_l2': 0.12938740107811092, 'drop_l3': 0.23993162405590604, 'drop_l4': 0.21576096316765284, 'drop_l5': 0.3835070986338437, 'drop_l6': 0.6068431812511641, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:41:38,393] Trial 412 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.10077469248049992, 'drop_l2': 0.11325658839707971, 'drop_l3': 0.21357433958554053, 'drop_l4': 0.20991730066333103, 'drop_l5': 0.2642936053005483, 'drop_l6': 0.6073068559474882, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:41:59,089] Trial 413 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.13717770688523812, 'drop_l2': 0.11538033603406038, 'drop_l3': 0.1404657594655533, 'drop_l4': 0.10102782020542543, 'drop_l5': 0.304539472557446, 'drop_l6': 0.7010640706397718, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:42:01,357] Trial 415 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.18709017178568485, 'drop_l2': 0.1368773902961909, 'drop_l3': 0.1365132856307004, 'drop_l4': 0.2292097925298891, 'drop_l5': 0.31431425052941936, 'drop_l6': 0.600090243703371, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:42:01,709] Trial 414 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.1434289976169142, 'drop_l2': 0.10102564791883459, 'drop_l3': 0.20969251576322182, 'drop_l4': 0.1028755217524621, 'drop_l5': 0.19281825459342633, 'drop_l6': 0.6027202033708378, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:42:12,516] Trial 416 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.17766596403236443, 'drop_l2': 0.1416433966930548, 'drop_l3': 0.11253859388180326, 'drop_l4': 0.19250943869501533, 'drop_l5': 0.2409941173206396, 'drop_l6': 0.5948582420897424, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:42:41,261] Trial 417 finished with value: 0.5843750238418579 and parameters: {'drop_l1': 0.17730435454388355, 'drop_l2': 0.4794428163813296, 'drop_l3': 0.10818939157388384, 'drop_l4': 0.10086658645513093, 'drop_l5': 0.23945858275954832, 'drop_l6': 0.626728133308283, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:42:43,804] Trial 419 finished with value: 0.625 and parameters: {'drop_l1': 0.12254785751054229, 'drop_l2': 0.14439589468867126, 'drop_l3': 0.1580932197510445, 'drop_l4': 0.19270757323079046, 'drop_l5': 0.2350842364584661, 'drop_l6': 0.6253168754350998, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:42:43,806] Trial 418 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.11541531545101175, 'drop_l2': 0.10183117573324722, 'drop_l3': 0.164890406196705, 'drop_l4': 0.28383138710197175, 'drop_l5': 0.6329586176513418, 'drop_l6': 0.5929061516193981, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:42:49,290] Trial 420 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.11427034992976397, 'drop_l2': 0.10068261668652097, 'drop_l3': 0.16302737529782596, 'drop_l4': 0.1643036027899811, 'drop_l5': 0.4950740776463309, 'drop_l6': 0.551346407602037, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:43:22,936] Trial 421 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.1209353131081498, 'drop_l2': 0.10072534294295181, 'drop_l3': 0.2778842170150856, 'drop_l4': 0.2755034301183126, 'drop_l5': 0.2909975148977378, 'drop_l6': 0.7745096728944478, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:43:27,174] Trial 423 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.10152731108211928, 'drop_l2': 0.1002500091430844, 'drop_l3': 0.27462166941135263, 'drop_l4': 0.17052554859112934, 'drop_l5': 0.4898453701529507, 'drop_l6': 0.8013354045938071, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:43:28,224] Trial 422 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.10065384542122378, 'drop_l2': 0.11427068510547564, 'drop_l3': 0.282022398047985, 'drop_l4': 0.17403493741994716, 'drop_l5': 0.5021571228919706, 'drop_l6': 0.7742187177714795, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:43:29,164] Trial 424 finished with value: 0.640625 and parameters: {'drop_l1': 0.15696448047963182, 'drop_l2': 0.16548286592744657, 'drop_l3': 0.17182358649924243, 'drop_l4': 0.22163619714033617, 'drop_l5': 0.5660029641306624, 'drop_l6': 0.6825603151182748, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:43:59,062] Trial 425 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.15340136807692428, 'drop_l2': 0.17201425928597527, 'drop_l3': 0.14269947471682376, 'drop_l4': 0.23837156368716828, 'drop_l5': 0.4135275089693145, 'drop_l6': 0.6835881067000236, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:44:09,654] Trial 428 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.13616467554540543, 'drop_l2': 0.1377764745454377, 'drop_l3': 0.21710521622013426, 'drop_l4': 0.19704477945670382, 'drop_l5': 0.6005478198156241, 'drop_l6': 0.6154736533794486, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:44:09,682] Trial 426 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.1349035067767903, 'drop_l2': 0.16499921766507158, 'drop_l3': 0.22046863834282193, 'drop_l4': 0.147466387192744, 'drop_l5': 0.5151301780479335, 'drop_l6': 0.6418765953616428, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:44:09,859] Trial 427 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.41817183945210185, 'drop_l2': 0.16760720437340865, 'drop_l3': 0.21506190773451989, 'drop_l4': 0.2318897887466906, 'drop_l5': 0.5671255301484491, 'drop_l6': 0.6748200675088916, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:44:29,229] Trial 429 finished with value: 0.609375 and parameters: {'drop_l1': 0.13737409358051011, 'drop_l2': 0.13267388123234164, 'drop_l3': 0.15050334595874063, 'drop_l4': 0.11741655902138388, 'drop_l5': 0.5207471586716834, 'drop_l6': 0.6434049304167937, 'activation': 'tanh', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:44:49,965] Trial 431 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.12805317124105942, 'drop_l2': 0.15074260259099612, 'drop_l3': 0.1988844699569929, 'drop_l4': 0.2173193939037608, 'drop_l5': 0.5468563267804393, 'drop_l6': 0.5700666227385275, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:44:50,840] Trial 430 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.13514219811384004, 'drop_l2': 0.1478327061396802, 'drop_l3': 0.19460510190257985, 'drop_l4': 0.21415726741009045, 'drop_l5': 0.5483716381110443, 'drop_l6': 0.5678847446494226, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:44:50,863] Trial 432 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.11316834056942615, 'drop_l2': 0.13472120559052186, 'drop_l3': 0.1749315598330728, 'drop_l4': 0.11878844237720868, 'drop_l5': 0.5484510595171384, 'drop_l6': 0.7159405577781961, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:45:01,655] Trial 433 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.10034817566020986, 'drop_l2': 0.14926206305537787, 'drop_l3': 0.12163124002967912, 'drop_l4': 0.21397768429006672, 'drop_l5': 0.20522439618931637, 'drop_l6': 0.5735399206685219, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:45:30,529] Trial 434 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1600719673734008, 'drop_l2': 0.11599360839424921, 'drop_l3': 0.17237601591213247, 'drop_l4': 0.24814827689431548, 'drop_l5': 0.5828210791172355, 'drop_l6': 0.6188653211289641, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:45:32,016] Trial 435 finished with value: 0.65625 and parameters: {'drop_l1': 0.11360159160260637, 'drop_l2': 0.1157090315392483, 'drop_l3': 0.25618156442273676, 'drop_l4': 0.15571909851324012, 'drop_l5': 0.34312490528608464, 'drop_l6': 0.6529900364162284, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:45:32,088] Trial 436 finished with value: 0.5718749761581421 and parameters: {'drop_l1': 0.10196962619541881, 'drop_l2': 0.1175881227499226, 'drop_l3': 0.1231544623390673, 'drop_l4': 0.15478919298875127, 'drop_l5': 0.27378049180388603, 'drop_l6': 0.6566646391056248, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:45:39,296] Trial 437 finished with value: 0.65625 and parameters: {'drop_l1': 0.1136383803870959, 'drop_l2': 0.11580303196805802, 'drop_l3': 0.12261042004370223, 'drop_l4': 0.1980100695612384, 'drop_l5': 0.19104985170935956, 'drop_l6': 0.5382155028534422, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:46:10,297] Trial 438 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.10071929709732041, 'drop_l2': 0.10013467123095092, 'drop_l3': 0.11788742131773285, 'drop_l4': 0.15689840593899176, 'drop_l5': 0.21883235321504424, 'drop_l6': 0.5181264333221255, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:46:14,045] Trial 439 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.10111719601037443, 'drop_l2': 0.1134096660159459, 'drop_l3': 0.26142546357647534, 'drop_l4': 0.2340756187354683, 'drop_l5': 0.17567746176991317, 'drop_l6': 0.6612534478183417, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:46:14,363] Trial 440 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.11550651821634858, 'drop_l2': 0.45458298212871673, 'drop_l3': 0.10045532672821406, 'drop_l4': 0.25659612229584966, 'drop_l5': 0.21225036914387638, 'drop_l6': 0.6528774623793785, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:46:16,191] Trial 441 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.10174992258540355, 'drop_l2': 0.10093804706845924, 'drop_l3': 0.10203626781152877, 'drop_l4': 0.19983859903032475, 'drop_l5': 0.1473887210687393, 'drop_l6': 0.5106738841602703, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:46:50,422] Trial 442 finished with value: 0.609375 and parameters: {'drop_l1': 0.16920650527210213, 'drop_l2': 0.4128024649924908, 'drop_l3': 0.10000909187402406, 'drop_l4': 0.23263089640057485, 'drop_l5': 0.17387097945975444, 'drop_l6': 0.5470373799244872, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:46:57,258] Trial 444 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.11256638139517096, 'drop_l2': 0.41717893006291695, 'drop_l3': 0.11779545580114323, 'drop_l4': 0.26738426035085977, 'drop_l5': 0.4686565522722788, 'drop_l6': 0.5417158153857418, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:46:58,181] Trial 445 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.1684763906102129, 'drop_l2': 0.45617808102755936, 'drop_l3': 0.11184972285417641, 'drop_l4': 0.26827241008370584, 'drop_l5': 0.2066742092414176, 'drop_l6': 0.5820815727567767, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:46:58,430] Trial 443 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.11741068552141344, 'drop_l2': 0.46193666540753797, 'drop_l3': 0.10155051634525512, 'drop_l4': 0.2106878392465177, 'drop_l5': 0.18889194139487397, 'drop_l6': 0.5278159526541663, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:47:22,856] Trial 446 finished with value: 0.640625 and parameters: {'drop_l1': 0.11869711823944258, 'drop_l2': 0.10041352427012315, 'drop_l3': 0.10085514057800142, 'drop_l4': 0.25502731432959314, 'drop_l5': 0.3633974020270111, 'drop_l6': 0.6914703315422663, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:47:40,471] Trial 449 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.14536847845840103, 'drop_l2': 0.47035794970804873, 'drop_l3': 0.11113961927600258, 'drop_l4': 0.21416747620889443, 'drop_l5': 0.192789881549351, 'drop_l6': 0.5314744812493162, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:47:40,572] Trial 448 finished with value: 0.640625 and parameters: {'drop_l1': 0.12673325153275228, 'drop_l2': 0.23692387731463982, 'drop_l3': 0.13320136123720683, 'drop_l4': 0.21263195924709552, 'drop_l5': 0.2064411337984135, 'drop_l6': 0.5552467823513572, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:47:40,706] Trial 447 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.19522413430251517, 'drop_l2': 0.49482724977738085, 'drop_l3': 0.10471525741552108, 'drop_l4': 0.254777480691057, 'drop_l5': 0.15347818637068653, 'drop_l6': 0.5311748588368121, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:47:53,508] Trial 450 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.14604895362251713, 'drop_l2': 0.39530738914995095, 'drop_l3': 0.10476015793218085, 'drop_l4': 0.2566320656197038, 'drop_l5': 0.19758225250944628, 'drop_l6': 0.5569910584779426, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:48:22,207] Trial 452 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.122744627121784, 'drop_l2': 0.704445005301331, 'drop_l3': 0.10033141602461199, 'drop_l4': 0.1964689530240143, 'drop_l5': 0.20353744008285665, 'drop_l6': 0.5045534043812392, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:48:23,509] Trial 451 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.19283810980879693, 'drop_l2': 0.44833733634401274, 'drop_l3': 0.1025029527611705, 'drop_l4': 0.2525346264895274, 'drop_l5': 0.1753379096519185, 'drop_l6': 0.5630509895771239, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:48:23,986] Trial 453 finished with value: 0.640625 and parameters: {'drop_l1': 0.11699902013022251, 'drop_l2': 0.12786418608798028, 'drop_l3': 0.13676875694442778, 'drop_l4': 0.18221234031708639, 'drop_l5': 0.1916651408000782, 'drop_l6': 0.4935226720266216, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:48:30,086] Trial 454 finished with value: 0.59375 and parameters: {'drop_l1': 0.10121370064405162, 'drop_l2': 0.7938520797669713, 'drop_l3': 0.1345790668312022, 'drop_l4': 0.18514965728804061, 'drop_l5': 0.18551605629945822, 'drop_l6': 0.11648860625174584, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:49:06,485] Trial 455 finished with value: 0.65625 and parameters: {'drop_l1': 0.10023396550660248, 'drop_l2': 0.5209103643667388, 'drop_l3': 0.13100519797242985, 'drop_l4': 0.183283701391216, 'drop_l5': 0.17043778835833995, 'drop_l6': 0.529864987958204, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:49:06,497] Trial 456 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.15374812568441412, 'drop_l2': 0.4644745179915306, 'drop_l3': 0.32850342556659695, 'drop_l4': 0.22771766695757473, 'drop_l5': 0.18966976163103388, 'drop_l6': 0.5860236050529114, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:49:06,695] Trial 457 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.11456858558958047, 'drop_l2': 0.5039736578473655, 'drop_l3': 0.14060927497206516, 'drop_l4': 0.23055474618475316, 'drop_l5': 0.1856174559024458, 'drop_l6': 0.5198533543891598, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:49:09,033] Trial 458 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.11513380655218879, 'drop_l2': 0.48467351132373543, 'drop_l3': 0.12450992416245868, 'drop_l4': 0.23626773058764156, 'drop_l5': 0.178821000426038, 'drop_l6': 0.5425279302929092, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:49:52,352] Trial 462 finished with value: 0.609375 and parameters: {'drop_l1': 0.5864211255196115, 'drop_l2': 0.4364638581284794, 'drop_l3': 0.43355270063367546, 'drop_l4': 0.20893415105959667, 'drop_l5': 0.2169617349393954, 'drop_l6': 0.5695738670059716, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:49:52,707] Trial 461 finished with value: 0.609375 and parameters: {'drop_l1': 0.10090019946215442, 'drop_l2': 0.44281234822450594, 'drop_l3': 0.1220578692068893, 'drop_l4': 0.19885913784915235, 'drop_l5': 0.1407269381035245, 'drop_l6': 0.5219177302541677, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:49:53,330] Trial 460 finished with value: 0.6625000238418579 and parameters: {'drop_l1': 0.10008114757669746, 'drop_l2': 0.11685913781944092, 'drop_l3': 0.11908680449230863, 'drop_l4': 0.2100559494932322, 'drop_l5': 0.3198784875587222, 'drop_l6': 0.5393256416751727, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:49:53,340] Trial 459 finished with value: 0.640625 and parameters: {'drop_l1': 0.11623643418051437, 'drop_l2': 0.48634797333443625, 'drop_l3': 0.12381051076836966, 'drop_l4': 0.20571555761523752, 'drop_l5': 0.3482337047847141, 'drop_l6': 0.5305664639499593, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:50:35,163] Trial 463 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.13483362046075534, 'drop_l2': 0.4575249777929019, 'drop_l3': 0.2563793961474036, 'drop_l4': 0.28874575522088525, 'drop_l5': 0.22648416462835314, 'drop_l6': 0.6412872870385878, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:50:36,657] Trial 464 finished with value: 0.609375 and parameters: {'drop_l1': 0.1001487392920602, 'drop_l2': 0.47330186428611437, 'drop_l3': 0.25605050729627843, 'drop_l4': 0.21425105594607008, 'drop_l5': 0.22475036226831513, 'drop_l6': 0.6379109904747342, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:50:36,933] Trial 466 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.1016749511124344, 'drop_l2': 0.4772120488223277, 'drop_l3': 0.10279513481561875, 'drop_l4': 0.22237677293037156, 'drop_l5': 0.17252995727834022, 'drop_l6': 0.5013963265311592, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:50:37,089] Trial 465 finished with value: 0.5874999761581421 and parameters: {'drop_l1': 0.13214701867954703, 'drop_l2': 0.4492945900580084, 'drop_l3': 0.11726423729954123, 'drop_l4': 0.21728943999803976, 'drop_l5': 0.20780769168537983, 'drop_l6': 0.5153761584329741, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:51:14,969] Trial 467 finished with value: 0.6625000238418579 and parameters: {'drop_l1': 0.10266438891680967, 'drop_l2': 0.11724593778827817, 'drop_l3': 0.10064946751629307, 'drop_l4': 0.22369412038128203, 'drop_l5': 0.20991312028178322, 'drop_l6': 0.5387642301851462, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:51:17,409] Trial 470 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.11593435616769432, 'drop_l2': 0.1127340038999645, 'drop_l3': 0.10029770152995823, 'drop_l4': 0.2393762556416444, 'drop_l5': 0.20982657325335785, 'drop_l6': 0.697092929312878, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:51:18,814] Trial 469 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.1281789550860588, 'drop_l2': 0.1453487228890289, 'drop_l3': 0.2042787749933331, 'drop_l4': 0.23961557951229126, 'drop_l5': 0.3016907109475257, 'drop_l6': 0.5529037243052738, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:51:18,955] Trial 468 finished with value: 0.675000011920929 and parameters: {'drop_l1': 0.13230778048372732, 'drop_l2': 0.1409672816744652, 'drop_l3': 0.1536149076756146, 'drop_l4': 0.2237062809021317, 'drop_l5': 0.3207503448185762, 'drop_l6': 0.7593199932379339, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:51:54,830] Trial 471 finished with value: 0.640625 and parameters: {'drop_l1': 0.10442030894992048, 'drop_l2': 0.5367210338889933, 'drop_l3': 0.10565701090164425, 'drop_l4': 0.182595792727107, 'drop_l5': 0.17460593877793362, 'drop_l6': 0.5332357966739727, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:51:56,572] Trial 472 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.14343805297161752, 'drop_l2': 0.4329672831824124, 'drop_l3': 0.10880374443124972, 'drop_l4': 0.2316387672690346, 'drop_l5': 0.20904237038467427, 'drop_l6': 0.7234481608081768, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:51:59,494] Trial 473 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.14334889920661792, 'drop_l2': 0.10168252285941017, 'drop_l3': 0.10844710046122161, 'drop_l4': 0.23991946932014055, 'drop_l5': 0.20200209697767366, 'drop_l6': 0.7021650709579457, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:51:59,634] Trial 474 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.13997763033638, 'drop_l2': 0.1345214990832162, 'drop_l3': 0.7492262246869819, 'drop_l4': 0.22874214637906898, 'drop_l5': 0.20259347659848642, 'drop_l6': 0.7560743888850507, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:52:35,795] Trial 475 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.15019384965865853, 'drop_l2': 0.10104619876209033, 'drop_l3': 0.10725847260094204, 'drop_l4': 0.2422188546141943, 'drop_l5': 0.3245566897317606, 'drop_l6': 0.7204364244785357, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:52:37,760] Trial 476 finished with value: 0.640625 and parameters: {'drop_l1': 0.12975346460213552, 'drop_l2': 0.12843524360180014, 'drop_l3': 0.10008741480395368, 'drop_l4': 0.22418371173833138, 'drop_l5': 0.3359051039651298, 'drop_l6': 0.7520743952917721, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:52:39,768] Trial 477 finished with value: 0.675000011920929 and parameters: {'drop_l1': 0.13062788781648146, 'drop_l2': 0.10036023584538148, 'drop_l3': 0.14972458052322393, 'drop_l4': 0.22208438422233182, 'drop_l5': 0.3267252330413011, 'drop_l6': 0.7409688247058991, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:52:40,156] Trial 478 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.12527437160920657, 'drop_l2': 0.11608736712546026, 'drop_l3': 0.15038909158125635, 'drop_l4': 0.22269759081755244, 'drop_l5': 0.2178864164335303, 'drop_l6': 0.7371195176914113, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:53:15,793] Trial 479 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.12278920842612645, 'drop_l2': 0.18686166346166616, 'drop_l3': 0.1002161685187832, 'drop_l4': 0.21522343433431756, 'drop_l5': 0.2323744307129975, 'drop_l6': 0.5457224209113605, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:53:19,124] Trial 480 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.11644420439505365, 'drop_l2': 0.17742416226923302, 'drop_l3': 0.15032791489835512, 'drop_l4': 0.21479595356233197, 'drop_l5': 0.21826558898349258, 'drop_l6': 0.7387937032259366, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:53:20,865] Trial 481 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.11853782346275799, 'drop_l2': 0.10010457679769041, 'drop_l3': 0.14784549848208164, 'drop_l4': 0.21241462483043233, 'drop_l5': 0.3137434594118818, 'drop_l6': 0.7641466527385556, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:53:21,133] Trial 482 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.1151019754446053, 'drop_l2': 0.1000048686774947, 'drop_l3': 0.1299138637304581, 'drop_l4': 0.21314011146134587, 'drop_l5': 0.3276655098297143, 'drop_l6': 0.74419558126295, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:53:52,822] Trial 483 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.11632354604271135, 'drop_l2': 0.675076068865668, 'drop_l3': 0.12547510039388152, 'drop_l4': 0.24660810985666234, 'drop_l5': 0.33018727511156787, 'drop_l6': 0.7637555686209289, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:54:00,613] Trial 486 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.10147277013779513, 'drop_l2': 0.11548524986132441, 'drop_l3': 0.12371781544283156, 'drop_l4': 0.24578856446549208, 'drop_l5': 0.3214952049337525, 'drop_l6': 0.7091586066029116, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:54:00,827] Trial 485 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.10168905537779831, 'drop_l2': 0.11285982319174767, 'drop_l3': 0.12941897858928575, 'drop_l4': 0.24610330502739572, 'drop_l5': 0.36110847463950346, 'drop_l6': 0.7324293863063178, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:54:01,518] Trial 484 finished with value: 0.625 and parameters: {'drop_l1': 0.10045565599746054, 'drop_l2': 0.11543470539936065, 'drop_l3': 0.13082036652176238, 'drop_l4': 0.25089400833207604, 'drop_l5': 0.32753921859773405, 'drop_l6': 0.7584932273391751, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:54:20,902] Trial 487 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.10037723781528993, 'drop_l2': 0.8652934763232447, 'drop_l3': 0.12031815726132572, 'drop_l4': 0.23913677448670081, 'drop_l5': 0.3130962768442765, 'drop_l6': 0.7021835915797111, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:54:41,450] Trial 488 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.16710013134987706, 'drop_l2': 0.13002382911172686, 'drop_l3': 0.15367414967851642, 'drop_l4': 0.23247414344029796, 'drop_l5': 0.33876668734447785, 'drop_l6': 0.7154358355135841, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:54:41,451] Trial 490 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.10032240870007027, 'drop_l2': 0.1295007334946062, 'drop_l3': 0.5600797968226202, 'drop_l4': 0.26567595441285796, 'drop_l5': 0.367858977490002, 'drop_l6': 0.7303345056224663, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:54:41,511] Trial 489 finished with value: 0.671875 and parameters: {'drop_l1': 0.1004922135267154, 'drop_l2': 0.11821717371428594, 'drop_l3': 0.11612712940497885, 'drop_l4': 0.2644815939014319, 'drop_l5': 0.36925286461617884, 'drop_l6': 0.7315242702881776, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:54:52,428] Trial 491 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.16349577341835342, 'drop_l2': 0.12954450920978983, 'drop_l3': 0.15164660572098465, 'drop_l4': 0.27020264997664767, 'drop_l5': 0.3641391650456315, 'drop_l6': 0.7174253242667509, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:55:20,657] Trial 492 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.15502592270881418, 'drop_l2': 0.1542332504834591, 'drop_l3': 0.10063369899451001, 'drop_l4': 0.2983470227610831, 'drop_l5': 0.3531598322784814, 'drop_l6': 0.7519214083510718, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:55:21,839] Trial 493 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.10057056326031037, 'drop_l2': 0.1543012221931297, 'drop_l3': 0.14332139581830888, 'drop_l4': 0.2793323320784649, 'drop_l5': 0.3502772988660898, 'drop_l6': 0.7317461736815069, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:55:21,852] Trial 494 finished with value: 0.625 and parameters: {'drop_l1': 0.1005187914441345, 'drop_l2': 0.10113517962282571, 'drop_l3': 0.10017259503302492, 'drop_l4': 0.26150578500943444, 'drop_l5': 0.3490257700682558, 'drop_l6': 0.7338581790751395, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:55:27,286] Trial 495 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.10001372373341483, 'drop_l2': 0.10023867161034217, 'drop_l3': 0.11491833912594289, 'drop_l4': 0.2858905350922022, 'drop_l5': 0.36559259292488017, 'drop_l6': 0.7371787743091689, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:56:03,020] Trial 496 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.1358230985963191, 'drop_l2': 0.10171111063724823, 'drop_l3': 0.11592061149643297, 'drop_l4': 0.2665063863016194, 'drop_l5': 0.37126864591564884, 'drop_l6': 0.7377693731162047, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:56:03,700] Trial 497 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1009751152524443, 'drop_l2': 0.13946984101319249, 'drop_l3': 0.11628170966552652, 'drop_l4': 0.24365955393945934, 'drop_l5': 0.39229315435386475, 'drop_l6': 0.7452795511674192, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:56:03,956] Trial 498 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.1335636489836857, 'drop_l2': 0.13983967900994243, 'drop_l3': 0.12406628839872624, 'drop_l4': 0.2528922843846976, 'drop_l5': 0.37438011605529514, 'drop_l6': 0.7235219691069115, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:56:05,863] Trial 499 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.8706404273019621, 'drop_l2': 0.14066481856589788, 'drop_l3': 0.13212270582293512, 'drop_l4': 0.2509343035919478, 'drop_l5': 0.37967135475134645, 'drop_l6': 0.7053157071118136, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:56:46,570] Trial 502 finished with value: 0.625 and parameters: {'drop_l1': 0.14293415673275991, 'drop_l2': 0.14285150329010607, 'drop_l3': 0.12085433391529764, 'drop_l4': 0.2617942356508464, 'drop_l5': 0.3749812090709407, 'drop_l6': 0.7198311423267533, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:56:46,615] Trial 500 finished with value: 0.625 and parameters: {'drop_l1': 0.1378751687423977, 'drop_l2': 0.13707100327828628, 'drop_l3': 0.1007023095287718, 'drop_l4': 0.23922154894878392, 'drop_l5': 0.38996485006316317, 'drop_l6': 0.7174076959672888, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:56:47,373] Trial 503 finished with value: 0.65625 and parameters: {'drop_l1': 0.14127150534572808, 'drop_l2': 0.11496690614703617, 'drop_l3': 0.12070289903115414, 'drop_l4': 0.27082849136319503, 'drop_l5': 0.39449319873041233, 'drop_l6': 0.7166920491314479, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:56:47,434] Trial 501 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.1158604206631789, 'drop_l2': 0.10074016417610408, 'drop_l3': 0.10198059038895099, 'drop_l4': 0.22989167651536407, 'drop_l5': 0.375012030514258, 'drop_l6': 0.7152270366611, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:57:26,856] Trial 504 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.21141601093921775, 'drop_l2': 0.11545303048249461, 'drop_l3': 0.13657736838876572, 'drop_l4': 0.23111639051419872, 'drop_l5': 0.3431062760030783, 'drop_l6': 0.7773143040220545, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:57:28,603] Trial 505 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.2218240356433786, 'drop_l2': 0.11379433880987981, 'drop_l3': 0.10154238297011939, 'drop_l4': 0.20441602213869767, 'drop_l5': 0.4237427807776024, 'drop_l6': 0.7659643912715225, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:57:28,652] Trial 506 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.11594361586571608, 'drop_l2': 0.11264285937620058, 'drop_l3': 0.13750513703477146, 'drop_l4': 0.2292475713512317, 'drop_l5': 0.42153157409371317, 'drop_l6': 0.7827610086642596, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:57:28,686] Trial 507 finished with value: 0.640625 and parameters: {'drop_l1': 0.11598695882780814, 'drop_l2': 0.11399493869209276, 'drop_l3': 0.14508394431190638, 'drop_l4': 0.22344113580053776, 'drop_l5': 0.3465789787416499, 'drop_l6': 0.7796425745114449, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:58:06,717] Trial 508 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.18459675834640213, 'drop_l2': 0.12059219898396602, 'drop_l3': 0.14879387822410944, 'drop_l4': 0.19934704861674038, 'drop_l5': 0.3969849017026276, 'drop_l6': 0.6923662078122093, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:58:11,351] Trial 511 finished with value: 0.565625011920929 and parameters: {'drop_l1': 0.21345106902185146, 'drop_l2': 0.11342456545684775, 'drop_l3': 0.10090303470857656, 'drop_l4': 0.203245563400732, 'drop_l5': 0.4088288443569306, 'drop_l6': 0.7590778423602511, 'activation': 'tanh', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:58:11,397] Trial 509 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.17691796252406763, 'drop_l2': 0.11492379307561384, 'drop_l3': 0.1000262865798146, 'drop_l4': 0.20423032397975283, 'drop_l5': 0.4130778883882289, 'drop_l6': 0.7726958381433314, 'activation': 'tanh', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:58:11,794] Trial 510 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.2304499347071428, 'drop_l2': 0.11625941792622292, 'drop_l3': 0.1006878154696739, 'drop_l4': 0.30840945772493744, 'drop_l5': 0.42863154258003555, 'drop_l6': 0.7617957210370143, 'activation': 'tanh', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:58:42,058] Trial 512 finished with value: 0.640625 and parameters: {'drop_l1': 0.17581686181064843, 'drop_l2': 0.10038368971904497, 'drop_l3': 0.1671412279147134, 'drop_l4': 0.19827143137238687, 'drop_l5': 0.4278073764505361, 'drop_l6': 0.7575188187047082, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:58:52,710] Trial 513 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.22512530773861597, 'drop_l2': 0.10045188868512452, 'drop_l3': 0.6015302641280654, 'drop_l4': 0.21649893030419992, 'drop_l5': 0.3368235745706464, 'drop_l6': 0.7463974468366527, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:58:52,767] Trial 515 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.13064928150015148, 'drop_l2': 0.159140485189517, 'drop_l3': 0.11572515854766041, 'drop_l4': 0.21787589025501286, 'drop_l5': 0.3540274161599274, 'drop_l6': 0.7483646161726657, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:58:52,773] Trial 514 finished with value: 0.640625 and parameters: {'drop_l1': 0.6686053344212822, 'drop_l2': 0.16214523083208776, 'drop_l3': 0.11713475512338525, 'drop_l4': 0.22491981487106824, 'drop_l5': 0.3598172956031477, 'drop_l6': 0.7429564452246256, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:59:10,349] Trial 516 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.6723875944296003, 'drop_l2': 0.12924741675482143, 'drop_l3': 0.11515457602960957, 'drop_l4': 0.21935864637127545, 'drop_l5': 0.1580095283938318, 'drop_l6': 0.7492103768856198, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:59:31,792] Trial 517 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.15452191083993252, 'drop_l2': 0.15719062295244918, 'drop_l3': 0.1599964485288808, 'drop_l4': 0.2802771653320194, 'drop_l5': 0.38749653593180317, 'drop_l6': 0.7912658339760434, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:59:32,861] Trial 519 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.15065965148560997, 'drop_l2': 0.720537311022531, 'drop_l3': 0.13098823201510948, 'drop_l4': 0.25100115492445446, 'drop_l5': 0.3571664178730658, 'drop_l6': 0.7891644937463514, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:59:33,773] Trial 518 finished with value: 0.625 and parameters: {'drop_l1': 0.15283381119307587, 'drop_l2': 0.1344481314092871, 'drop_l3': 0.13282735204697163, 'drop_l4': 0.19928017024064346, 'drop_l5': 0.3861946507019963, 'drop_l6': 0.7364428482319634, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 18:59:42,633] Trial 520 finished with value: 0.640625 and parameters: {'drop_l1': 0.10258256369942864, 'drop_l2': 0.1012104231095088, 'drop_l3': 0.1579192397382793, 'drop_l4': 0.24768426661543427, 'drop_l5': 0.29888189755816064, 'drop_l6': 0.8152554466009421, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:00:08,687] Trial 521 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.10034081399638826, 'drop_l2': 0.10152984345678151, 'drop_l3': 0.13701046423110397, 'drop_l4': 0.19374503913611046, 'drop_l5': 0.2466131534820494, 'drop_l6': 0.2979593841110043, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:00:11,543] Trial 522 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.1658278577788495, 'drop_l2': 0.13326077274308198, 'drop_l3': 0.13474367007006913, 'drop_l4': 0.24935101498050322, 'drop_l5': 0.38220768183669923, 'drop_l6': 0.7295261308105145, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:00:13,422] Trial 523 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1315247312280815, 'drop_l2': 0.12973291868393277, 'drop_l3': 0.13001340302830783, 'drop_l4': 0.19567317823760363, 'drop_l5': 0.3120495318510332, 'drop_l6': 0.7682758668041527, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:00:16,661] Trial 524 finished with value: 0.640625 and parameters: {'drop_l1': 0.1152938867952192, 'drop_l2': 0.1277720491808716, 'drop_l3': 0.13476763915037182, 'drop_l4': 0.2372947696531088, 'drop_l5': 0.6823261074838952, 'drop_l6': 0.4832974572117234, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:00:43,213] Trial 525 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.12947299354102007, 'drop_l2': 0.17914090624150864, 'drop_l3': 0.15634222479559262, 'drop_l4': 0.25382711260451374, 'drop_l5': 0.3708075519732541, 'drop_l6': 0.7303969989107372, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:00:50,788] Trial 527 finished with value: 0.6625000238418579 and parameters: {'drop_l1': 0.1266111176243126, 'drop_l2': 0.10068896274534306, 'drop_l3': 0.18380495082558385, 'drop_l4': 0.20752804962085517, 'drop_l5': 0.29620290095504404, 'drop_l6': 0.5489502136318072, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:00:50,859] Trial 526 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.2000826531749629, 'drop_l2': 0.12290563661334983, 'drop_l3': 0.10005792148935592, 'drop_l4': 0.21403056504244702, 'drop_l5': 0.44266894876075163, 'drop_l6': 0.5485508264348418, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:00:53,341] Trial 528 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.195550439938561, 'drop_l2': 0.12005342214580181, 'drop_l3': 0.1490011710570328, 'drop_l4': 0.22762444951171582, 'drop_l5': 0.407488062077734, 'drop_l6': 0.6938830870947439, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:01:13,767] Trial 529 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.17596829931147037, 'drop_l2': 0.14733567784023177, 'drop_l3': 0.10090184169483606, 'drop_l4': 0.6329914027402215, 'drop_l5': 0.4054225480263834, 'drop_l6': 0.5512450366678395, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:01:27,887] Trial 530 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.11932890047985313, 'drop_l2': 0.11705048497717255, 'drop_l3': 0.10022140541822953, 'drop_l4': 0.21731979414765515, 'drop_l5': 0.3006785238471164, 'drop_l6': 0.5132359423468682, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:01:28,810] Trial 531 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.12718562785121312, 'drop_l2': 0.11461678618507498, 'drop_l3': 0.11330103484959217, 'drop_l4': 0.27232195814494764, 'drop_l5': 0.3133726343244796, 'drop_l6': 0.52789567836791, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:01:29,697] Trial 532 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.18162939415154855, 'drop_l2': 0.10243558683291203, 'drop_l3': 0.18904273148575487, 'drop_l4': 0.21502306350382913, 'drop_l5': 0.313475043051272, 'drop_l6': 0.5192596259571663, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:01:47,705] Trial 533 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.10027150272399894, 'drop_l2': 0.1003580777032673, 'drop_l3': 0.18145140261008486, 'drop_l4': 0.5553531852566701, 'drop_l5': 0.28590422673769345, 'drop_l6': 0.5687445663332202, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:02:06,291] Trial 534 finished with value: 0.625 and parameters: {'drop_l1': 0.20643650366595545, 'drop_l2': 0.10219797867737353, 'drop_l3': 0.18450063699508398, 'drop_l4': 0.2767737622454679, 'drop_l5': 0.3318989867027755, 'drop_l6': 0.5627101795031304, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:02:07,824] Trial 535 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.20881093798632533, 'drop_l2': 0.10230171974707479, 'drop_l3': 0.229779512415111, 'drop_l4': 0.20496596053122823, 'drop_l5': 0.3209826493385351, 'drop_l6': 0.5662207430296715, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:02:08,281] Trial 536 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.1404650661769712, 'drop_l2': 0.10054400338298311, 'drop_l3': 0.22873980279993986, 'drop_l4': 0.25917267679409095, 'drop_l5': 0.3316840121502095, 'drop_l6': 0.7068448206047828, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:02:21,169] Trial 537 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.15644432429197952, 'drop_l2': 0.1566292753562295, 'drop_l3': 0.15853380651373533, 'drop_l4': 0.19426806652862372, 'drop_l5': 0.33674805926558343, 'drop_l6': 0.5394870436988503, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:02:48,269] Trial 538 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.15899964496014987, 'drop_l2': 0.11353344754063008, 'drop_l3': 0.15825164005550224, 'drop_l4': 0.2377620415758361, 'drop_l5': 0.33705060926951064, 'drop_l6': 0.6943361312654571, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:02:49,786] Trial 539 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.10104751693957648, 'drop_l2': 0.1486997820304855, 'drop_l3': 0.20413321984204785, 'drop_l4': 0.19444732766666342, 'drop_l5': 0.2936844935333972, 'drop_l6': 0.5439619505923791, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:02:49,791] Trial 540 finished with value: 0.65625 and parameters: {'drop_l1': 0.11502456246432827, 'drop_l2': 0.14518015842469956, 'drop_l3': 0.10065410703590476, 'drop_l4': 0.23726693159927917, 'drop_l5': 0.21503812127014765, 'drop_l6': 0.4980697339842637, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:02:55,913] Trial 541 finished with value: 0.609375 and parameters: {'drop_l1': 0.10914738329147292, 'drop_l2': 0.1476991051395457, 'drop_l3': 0.10188521052046516, 'drop_l4': 0.23418480151264168, 'drop_l5': 0.29662136311615134, 'drop_l6': 0.5050197466308866, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:03:31,363] Trial 542 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.13659595541822447, 'drop_l2': 0.14192169537998844, 'drop_l3': 0.12277034365676445, 'drop_l4': 0.20610329338572236, 'drop_l5': 0.36529775691947736, 'drop_l6': 0.8022082005099539, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:03:34,075] Trial 543 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.19660397122980477, 'drop_l2': 0.12712938920151406, 'drop_l3': 0.18121919367838601, 'drop_l4': 0.1885461098398422, 'drop_l5': 0.43077043404918436, 'drop_l6': 0.8007709188268445, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:03:34,079] Trial 544 finished with value: 0.574999988079071 and parameters: {'drop_l1': 0.14474278535766164, 'drop_l2': 0.5542077913178256, 'drop_l3': 0.17661663308264264, 'drop_l4': 0.18496146611754344, 'drop_l5': 0.4361275237633619, 'drop_l6': 0.18186520469335643, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:03:36,733] Trial 545 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.13525747991091502, 'drop_l2': 0.5592935707045297, 'drop_l3': 0.1193273077617795, 'drop_l4': 0.20609839801979962, 'drop_l5': 0.6286970407384489, 'drop_l6': 0.7972155018208618, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:04:14,305] Trial 546 finished with value: 0.6625000238418579 and parameters: {'drop_l1': 0.12843202111559912, 'drop_l2': 0.19518655004613977, 'drop_l3': 0.17829213288943327, 'drop_l4': 0.2527320011566679, 'drop_l5': 0.35050709858181045, 'drop_l6': 0.7295104412226319, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:04:16,749] Trial 547 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.10203045212043087, 'drop_l2': 0.10038975043356523, 'drop_l3': 0.14089127736740403, 'drop_l4': 0.22504976129872462, 'drop_l5': 0.24037052052940733, 'drop_l6': 0.7611574957661735, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:04:17,131] Trial 548 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.11360367161625895, 'drop_l2': 0.10103492165431062, 'drop_l3': 0.14213487678270184, 'drop_l4': 0.22921254034476024, 'drop_l5': 0.24396494126531354, 'drop_l6': 0.7676447056185772, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:04:18,384] Trial 549 finished with value: 0.640625 and parameters: {'drop_l1': 0.44423325816069054, 'drop_l2': 0.5791823118751661, 'drop_l3': 0.4604374256735191, 'drop_l4': 0.21693432646514932, 'drop_l5': 0.6131914053342511, 'drop_l6': 0.7801371842074968, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:04:54,669] Trial 550 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.13259724614183793, 'drop_l2': 0.2104325288928138, 'drop_l3': 0.11845867927021923, 'drop_l4': 0.2991709322005268, 'drop_l5': 0.6375026938277107, 'drop_l6': 0.8227997695929665, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:04:59,308] Trial 552 finished with value: 0.65625 and parameters: {'drop_l1': 0.6108873933528229, 'drop_l2': 0.57897103496728, 'drop_l3': 0.11531466959586616, 'drop_l4': 0.2928366078892016, 'drop_l5': 0.6542453787560568, 'drop_l6': 0.8160511780832058, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:05:00,162] Trial 553 finished with value: 0.625 and parameters: {'drop_l1': 0.133718161149252, 'drop_l2': 0.22575281265273128, 'drop_l3': 0.11530176155430628, 'drop_l4': 0.2092656908248983, 'drop_l5': 0.6197049992748193, 'drop_l6': 0.8100598858597117, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:05:00,169] Trial 551 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.1599755248724977, 'drop_l2': 0.19520509315023682, 'drop_l3': 0.47473980333208177, 'drop_l4': 0.2596383060151459, 'drop_l5': 0.6388006944333641, 'drop_l6': 0.8257628304347747, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:05:29,936] Trial 554 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.14475395870891236, 'drop_l2': 0.18198700114997204, 'drop_l3': 0.1625754903711921, 'drop_l4': 0.21326297669035868, 'drop_l5': 0.6687898896027344, 'drop_l6': 0.8049365822037241, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:05:42,234] Trial 555 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.12766166461740902, 'drop_l2': 0.19784660731248563, 'drop_l3': 0.1740356550113214, 'drop_l4': 0.262463017010179, 'drop_l5': 0.6291942621120516, 'drop_l6': 0.7936518921123631, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:05:42,320] Trial 557 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.12518493981515016, 'drop_l2': 0.5531667996044001, 'drop_l3': 0.20648504655739316, 'drop_l4': 0.200744164497307, 'drop_l5': 0.6603436112271084, 'drop_l6': 0.7870723720604536, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:05:42,341] Trial 556 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.18718756558326347, 'drop_l2': 0.6059109548055313, 'drop_l3': 0.21060353001651785, 'drop_l4': 0.20320867561754208, 'drop_l5': 0.3517676149986817, 'drop_l6': 0.7588058156218982, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:06:01,565] Trial 558 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.12291984149795819, 'drop_l2': 0.22335551231470177, 'drop_l3': 0.18506077473301044, 'drop_l4': 0.2284282162395193, 'drop_l5': 0.3472110166195383, 'drop_l6': 0.7472619211711619, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:06:24,048] Trial 559 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.12409780407218447, 'drop_l2': 0.5265581768121275, 'drop_l3': 0.19231460101033754, 'drop_l4': 0.24438501861569392, 'drop_l5': 0.606297969492082, 'drop_l6': 0.7305238661542821, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:06:25,526] Trial 561 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.14635854646981483, 'drop_l2': 0.5238874155112745, 'drop_l3': 0.12825234940972263, 'drop_l4': 0.24487432060373832, 'drop_l5': 0.3575305324144159, 'drop_l6': 0.7328901412427153, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:06:25,543] Trial 560 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.14700583256333655, 'drop_l2': 0.1687474071855938, 'drop_l3': 0.1238368349421545, 'drop_l4': 0.2530065216062504, 'drop_l5': 0.3600015283420996, 'drop_l6': 0.7350962226850254, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:06:34,272] Trial 562 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.1638742134618609, 'drop_l2': 0.167099665955217, 'drop_l3': 0.15571548658075948, 'drop_l4': 0.24533153623894738, 'drop_l5': 0.3307059022520726, 'drop_l6': 0.7295921676293138, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:07:05,690] Trial 564 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.1433177573901181, 'drop_l2': 0.134428529944981, 'drop_l3': 0.23256542937095132, 'drop_l4': 0.24322804032422785, 'drop_l5': 0.38274045522194455, 'drop_l6': 0.7213689486678614, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:07:05,694] Trial 563 finished with value: 0.625 and parameters: {'drop_l1': 0.15075175462178161, 'drop_l2': 0.13718369238109213, 'drop_l3': 0.13301430883881896, 'drop_l4': 0.24405270906739768, 'drop_l5': 0.41296476464185966, 'drop_l6': 0.5306768719235179, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:07:08,219] Trial 565 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.16385130255828598, 'drop_l2': 0.17132725396344958, 'drop_l3': 0.14609033389677606, 'drop_l4': 0.2861779015672299, 'drop_l5': 0.3151878739714781, 'drop_l6': 0.7229574103858677, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:07:10,853] Trial 566 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.1638451198712523, 'drop_l2': 0.1282119546431162, 'drop_l3': 0.8925587339798566, 'drop_l4': 0.23318707818379353, 'drop_l5': 0.3142069505660754, 'drop_l6': 0.6893746242948982, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:07:47,414] Trial 568 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.11409534347510139, 'drop_l2': 0.24698454288695262, 'drop_l3': 0.10165691142456433, 'drop_l4': 0.22471625674800635, 'drop_l5': 0.3179960101081853, 'drop_l6': 0.7691429054913678, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:07:47,647] Trial 567 finished with value: 0.625 and parameters: {'drop_l1': 0.10154733508998208, 'drop_l2': 0.11852138024106537, 'drop_l3': 0.6848896781461764, 'drop_l4': 0.22652747211663282, 'drop_l5': 0.32995132501068997, 'drop_l6': 0.7743143820463956, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:07:47,764] Trial 569 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.17428103700266762, 'drop_l2': 0.12605161019560626, 'drop_l3': 0.13455228813788594, 'drop_l4': 0.2598262213459827, 'drop_l5': 0.40074821666473026, 'drop_l6': 0.7635505274945167, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:07:49,741] Trial 570 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.18352053861434664, 'drop_l2': 0.2444090405266202, 'drop_l3': 0.22575490711849225, 'drop_l4': 0.1819848658790887, 'drop_l5': 0.40142787587218676, 'drop_l6': 0.7764978518278571, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:08:27,007] Trial 573 finished with value: 0.609375 and parameters: {'drop_l1': 0.10074467928005454, 'drop_l2': 0.14644400334193883, 'drop_l3': 0.8478176653557092, 'drop_l4': 0.26235370318403606, 'drop_l5': 0.3783134866926814, 'drop_l6': 0.7510407058700742, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:08:29,926] Trial 572 finished with value: 0.640625 and parameters: {'drop_l1': 0.17210778094552598, 'drop_l2': 0.11498833001939594, 'drop_l3': 0.22151926023767354, 'drop_l4': 0.18343520579939915, 'drop_l5': 0.3644351155243874, 'drop_l6': 0.7101766725098049, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:08:29,969] Trial 571 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.10032094190405491, 'drop_l2': 0.17499596012546972, 'drop_l3': 0.1648905248854553, 'drop_l4': 0.2675789258052331, 'drop_l5': 0.3348621044528975, 'drop_l6': 0.7101035311110819, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:08:30,024] Trial 574 finished with value: 0.59375 and parameters: {'drop_l1': 0.5024881349345437, 'drop_l2': 0.20696543822983018, 'drop_l3': 0.16740880007878023, 'drop_l4': 0.1793334231742225, 'drop_l5': 0.6381529509583397, 'drop_l6': 0.5571855052105497, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:09:07,378] Trial 575 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.11592576688313734, 'drop_l2': 0.10125606888530905, 'drop_l3': 0.24259909333650112, 'drop_l4': 0.20966186870577763, 'drop_l5': 0.6474471142327345, 'drop_l6': 0.5495502781300449, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:09:13,416] Trial 578 finished with value: 0.609375 and parameters: {'drop_l1': 0.10144942645320307, 'drop_l2': 0.11081684195452449, 'drop_l3': 0.19388385545930814, 'drop_l4': 0.20467952246614227, 'drop_l5': 0.2951141541403768, 'drop_l6': 0.5585324580382558, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:09:13,419] Trial 577 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.5051060058058665, 'drop_l2': 0.10020694588936735, 'drop_l3': 0.20784344774991592, 'drop_l4': 0.20056089683722844, 'drop_l5': 0.2716361335636456, 'drop_l6': 0.5677961263151408, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:09:13,476] Trial 576 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.18605639633586127, 'drop_l2': 0.15606750117364288, 'drop_l3': 0.1481461172312666, 'drop_l4': 0.20659009839215503, 'drop_l5': 0.27640851703696, 'drop_l6': 0.5732464168867573, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:09:39,647] Trial 579 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.13300700804621993, 'drop_l2': 0.15225137609201894, 'drop_l3': 0.10009613806241727, 'drop_l4': 0.2697025760875753, 'drop_l5': 0.3462567437904389, 'drop_l6': 0.7055605793210523, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:09:56,427] Trial 580 finished with value: 0.625 and parameters: {'drop_l1': 0.154128502230619, 'drop_l2': 0.12203062800814532, 'drop_l3': 0.180247302120553, 'drop_l4': 0.2286022636558843, 'drop_l5': 0.353266602770146, 'drop_l6': 0.7488930687077585, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:09:56,497] Trial 581 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.15530591968508772, 'drop_l2': 0.1625793923087283, 'drop_l3': 0.17683567184540291, 'drop_l4': 0.22275875850103463, 'drop_l5': 0.3404042507984223, 'drop_l6': 0.7492558210703556, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:09:56,531] Trial 582 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.15408191933277712, 'drop_l2': 0.16278645378256532, 'drop_l3': 0.17647200533845184, 'drop_l4': 0.22891467995081863, 'drop_l5': 0.311629197393436, 'drop_l6': 0.6881939440661408, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:10:11,547] Trial 583 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.12450497988775105, 'drop_l2': 0.6735111659621096, 'drop_l3': 0.11770456413271818, 'drop_l4': 0.22651813199604395, 'drop_l5': 0.6239275464379029, 'drop_l6': 0.6825729888918216, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:10:38,761] Trial 584 finished with value: 0.640625 and parameters: {'drop_l1': 0.11752208090200725, 'drop_l2': 0.1334112303134975, 'drop_l3': 0.10009325071761321, 'drop_l4': 0.21886825185270178, 'drop_l5': 0.3396977998719917, 'drop_l6': 0.5306688702987041, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:10:38,772] Trial 586 finished with value: 0.625 and parameters: {'drop_l1': 0.12986758617020147, 'drop_l2': 0.13138194168440365, 'drop_l3': 0.1142909613403414, 'drop_l4': 0.22096718323687403, 'drop_l5': 0.16214133570896908, 'drop_l6': 0.5224411935415568, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:10:39,168] Trial 585 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.11770764198214745, 'drop_l2': 0.49948734045035387, 'drop_l3': 0.15406203699020238, 'drop_l4': 0.23907908990256263, 'drop_l5': 0.16288487880380298, 'drop_l6': 0.5392147182612614, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:10:46,663] Trial 587 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.10000242856583388, 'drop_l2': 0.12805325570084863, 'drop_l3': 0.15993999259574157, 'drop_l4': 0.1719697988399717, 'drop_l5': 0.2747247135828779, 'drop_l6': 0.5816940094881834, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:11:20,728] Trial 589 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.19993175580350164, 'drop_l2': 0.5085125895904712, 'drop_l3': 0.14686724811694185, 'drop_l4': 0.20836546963993968, 'drop_l5': 0.6156673205556658, 'drop_l6': 0.5428434390382291, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:11:20,764] Trial 588 finished with value: 0.565625011920929 and parameters: {'drop_l1': 0.1352959826574817, 'drop_l2': 0.11900626984358718, 'drop_l3': 0.14606370365139407, 'drop_l4': 0.20205870060590228, 'drop_l5': 0.6016349470850302, 'drop_l6': 0.7913480247262887, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:11:21,218] Trial 590 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.13912507210479547, 'drop_l2': 0.14175913995207992, 'drop_l3': 0.13162277744465684, 'drop_l4': 0.27631437166711353, 'drop_l5': 0.3758608323151567, 'drop_l6': 0.7397508248714828, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:11:22,676] Trial 591 finished with value: 0.574999988079071 and parameters: {'drop_l1': 0.14202242223634987, 'drop_l2': 0.11065327194740625, 'drop_l3': 0.14664394794447744, 'drop_l4': 0.27864318546669603, 'drop_l5': 0.34060354966544576, 'drop_l6': 0.5448445658194125, 'activation': 'tanh', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:12:03,740] Trial 593 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.10019938227176374, 'drop_l2': 0.14625478771058983, 'drop_l3': 0.1644480656331367, 'drop_l4': 0.27875169634338165, 'drop_l5': 0.3194013281417546, 'drop_l6': 0.6988863816064301, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:12:03,920] Trial 595 finished with value: 0.609375 and parameters: {'drop_l1': 0.1630999010022392, 'drop_l2': 0.100538631172649, 'drop_l3': 0.16890630051459538, 'drop_l4': 0.1897386777591005, 'drop_l5': 0.31820451134133, 'drop_l6': 0.7007772056541549, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:12:04,995] Trial 594 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.10108893006970346, 'drop_l2': 0.18425732006762596, 'drop_l3': 0.12326018059411796, 'drop_l4': 0.31262516330533524, 'drop_l5': 0.39004014953918664, 'drop_l6': 0.7269456094481784, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:12:05,006] Trial 592 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.1356099131881785, 'drop_l2': 0.18081731280984065, 'drop_l3': 0.19385344574969263, 'drop_l4': 0.24798815690269696, 'drop_l5': 0.3195889457551717, 'drop_l6': 0.7033575288090674, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:12:44,999] Trial 596 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.22445957159723634, 'drop_l2': 0.11756639570375932, 'drop_l3': 0.13300649287305483, 'drop_l4': 0.2550522094670072, 'drop_l5': 0.3036347491711753, 'drop_l6': 0.7602224243764762, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:12:46,047] Trial 599 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.18272681376618724, 'drop_l2': 0.100245195139024, 'drop_l3': 0.14440826892160996, 'drop_l4': 0.19201102759119526, 'drop_l5': 0.4191939384011616, 'drop_l6': 0.5771691945492456, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:12:46,674] Trial 598 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.2187671307871392, 'drop_l2': 0.12474660226345408, 'drop_l3': 0.14228142014552805, 'drop_l4': 0.17699340814657016, 'drop_l5': 0.4408580054432042, 'drop_l6': 0.5803407724924888, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:12:46,846] Trial 597 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.10037654713738506, 'drop_l2': 0.10998001842896915, 'drop_l3': 0.20029914858609194, 'drop_l4': 0.24275556504810894, 'drop_l5': 0.36045117704240176, 'drop_l6': 0.7583205501350845, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:13:26,101] Trial 601 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.11553959916187852, 'drop_l2': 0.11573586428182647, 'drop_l3': 0.10131303831896155, 'drop_l4': 0.23918948311607896, 'drop_l5': 0.5887657541710339, 'drop_l6': 0.678038205161517, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:13:26,495] Trial 602 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.12590628779065002, 'drop_l2': 0.7027002978287635, 'drop_l3': 0.11613717982252797, 'drop_l4': 0.2564105017883858, 'drop_l5': 0.376481264823535, 'drop_l6': 0.7193157675685469, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:13:26,603] Trial 600 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.11455136140347595, 'drop_l2': 0.15708090936687738, 'drop_l3': 0.1924452902162325, 'drop_l4': 0.23798580444837447, 'drop_l5': 0.34882285518499334, 'drop_l6': 0.7889640078959079, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:13:26,664] Trial 603 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.15180674666748062, 'drop_l2': 0.14099459508425716, 'drop_l3': 0.11828196258578855, 'drop_l4': 0.25343519875186704, 'drop_l5': 0.3681402881510555, 'drop_l6': 0.717709423519701, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:14:05,561] Trial 604 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.11586362339951067, 'drop_l2': 0.1007718103128852, 'drop_l3': 0.11563346207521408, 'drop_l4': 0.2106286426193794, 'drop_l5': 0.19373243404523668, 'drop_l6': 0.5117898108070694, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:14:05,751] Trial 606 finished with value: 0.609375 and parameters: {'drop_l1': 0.12801471583679425, 'drop_l2': 0.13437515727584648, 'drop_l3': 0.22047832387776184, 'drop_l4': 0.18091692605034648, 'drop_l5': 0.6134117889160793, 'drop_l6': 0.7728420504464969, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:14:07,216] Trial 607 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.10020191981612449, 'drop_l2': 0.18696376228297026, 'drop_l3': 0.19855040329500453, 'drop_l4': 0.18206125818351807, 'drop_l5': 0.5966599474830973, 'drop_l6': 0.7760830904920831, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:14:07,285] Trial 605 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.17146564694685512, 'drop_l2': 0.10074860020129021, 'drop_l3': 0.12823694519895654, 'drop_l4': 0.2364324380210149, 'drop_l5': 0.28307863080895734, 'drop_l6': 0.5881480156545049, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:14:45,789] Trial 611 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.13010510483128165, 'drop_l2': 0.13966370929658403, 'drop_l3': 0.17441577113382206, 'drop_l4': 0.2916649385605395, 'drop_l5': 0.3838177918759873, 'drop_l6': 0.735653118913077, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:14:46,372] Trial 609 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.11325128227070758, 'drop_l2': 0.6438180178762174, 'drop_l3': 0.16519209068665344, 'drop_l4': 0.1932655518988838, 'drop_l5': 0.5707321680050388, 'drop_l6': 0.558920921703032, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:14:47,169] Trial 608 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.11639384231272742, 'drop_l2': 0.13106533882289456, 'drop_l3': 0.10289269327341166, 'drop_l4': 0.2195636682674377, 'drop_l5': 0.19998512920888084, 'drop_l6': 0.5226471774517483, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:14:47,404] Trial 610 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1336490760574908, 'drop_l2': 0.14039771919366822, 'drop_l3': 0.1778710775394741, 'drop_l4': 0.21642708448297698, 'drop_l5': 0.3844512433964528, 'drop_l6': 0.7497494293021911, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:15:26,384] Trial 613 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.1402492789874941, 'drop_l2': 0.14682757032753513, 'drop_l3': 0.10040754065901578, 'drop_l4': 0.21754866338266193, 'drop_l5': 0.3573328255880901, 'drop_l6': 0.4810516903555652, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:15:26,809] Trial 615 finished with value: 0.65625 and parameters: {'drop_l1': 0.10082869983119308, 'drop_l2': 0.11656989852297236, 'drop_l3': 0.21204325699280413, 'drop_l4': 0.2036775394982269, 'drop_l5': 0.37046472546574255, 'drop_l6': 0.7452377884561894, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:15:27,949] Trial 612 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.14512140968551054, 'drop_l2': 0.11867235698377977, 'drop_l3': 0.10137340689705387, 'drop_l4': 0.22666703528863025, 'drop_l5': 0.4225229680296708, 'drop_l6': 0.5632435567638476, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:15:28,013] Trial 614 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.10052548928030783, 'drop_l2': 0.1491484564370672, 'drop_l3': 0.1299358015561382, 'drop_l4': 0.22131064239523626, 'drop_l5': 0.3619770893955013, 'drop_l6': 0.7429546305566977, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:16:07,002] Trial 616 finished with value: 0.640625 and parameters: {'drop_l1': 0.10046147904223995, 'drop_l2': 0.10105302878063084, 'drop_l3': 0.1555464391524821, 'drop_l4': 0.35190269355653825, 'drop_l5': 0.33710671310857354, 'drop_l6': 0.5712556266055241, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:16:08,016] Trial 617 finished with value: 0.640625 and parameters: {'drop_l1': 0.11415851277854558, 'drop_l2': 0.12471453485057801, 'drop_l3': 0.18314524802364668, 'drop_l4': 0.16935810453672576, 'drop_l5': 0.2642946247969343, 'drop_l6': 0.5644730639776299, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:16:08,980] Trial 619 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.5501753657825948, 'drop_l2': 0.19677572382218783, 'drop_l3': 0.15950401491587243, 'drop_l4': 0.24284723470993094, 'drop_l5': 0.3295314010975665, 'drop_l6': 0.8015731654156786, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:16:09,755] Trial 618 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.553201012799225, 'drop_l2': 0.17240497636003263, 'drop_l3': 0.15754969978799638, 'drop_l4': 0.23801429053090042, 'drop_l5': 0.3376233509923445, 'drop_l6': 0.23061089919824762, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:16:50,121] Trial 620 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.12648832935997897, 'drop_l2': 0.12777353194061314, 'drop_l3': 0.1180495088067014, 'drop_l4': 0.25820442172655494, 'drop_l5': 0.3996745720446202, 'drop_l6': 0.724356938803587, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:16:52,295] Trial 622 finished with value: 0.625 and parameters: {'drop_l1': 0.10005739149394931, 'drop_l2': 0.10017076684768826, 'drop_l3': 0.1855738280228641, 'drop_l4': 0.19334846214903043, 'drop_l5': 0.22735181394150755, 'drop_l6': 0.5888128701629778, 'activation': 'relu', 'kernel_init': 'he_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:16:52,826] Trial 623 finished with value: 0.581250011920929 and parameters: {'drop_l1': 0.12071199883210933, 'drop_l2': 0.10127862201999793, 'drop_l3': 0.11374288057657998, 'drop_l4': 0.1984080680098692, 'drop_l5': 0.19066898794331782, 'drop_l6': 0.5350207226916212, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:16:52,844] Trial 621 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.1532135819328475, 'drop_l2': 0.3679790401606222, 'drop_l3': 0.15346743792740938, 'drop_l4': 0.23579550978197927, 'drop_l5': 0.18796781426995465, 'drop_l6': 0.5439126215182712, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:17:32,612] Trial 624 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.10005123466635982, 'drop_l2': 0.10152463552204635, 'drop_l3': 0.1905042650975516, 'drop_l4': 0.2049855246046697, 'drop_l5': 0.2232291871825175, 'drop_l6': 0.5916411588968958, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:17:37,352] Trial 625 finished with value: 0.640625 and parameters: {'drop_l1': 0.1133920977799881, 'drop_l2': 0.11272038141646697, 'drop_l3': 0.223546684533722, 'drop_l4': 0.19649712099962177, 'drop_l5': 0.21817277005721783, 'drop_l6': 0.7684488770970508, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:17:37,532] Trial 627 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.17298669095671376, 'drop_l2': 0.11609879526908072, 'drop_l3': 0.13955931257888357, 'drop_l4': 0.21359711590607525, 'drop_l5': 0.4156146921097522, 'drop_l6': 0.7647848142785024, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:17:37,560] Trial 626 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.11385581573451753, 'drop_l2': 0.11812324103420525, 'drop_l3': 0.14048589781274776, 'drop_l4': 0.21050187587622168, 'drop_l5': 0.2937395259842911, 'drop_l6': 0.5772530913905645, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:18:06,614] Trial 628 finished with value: 0.675000011920929 and parameters: {'drop_l1': 0.10037909456342504, 'drop_l2': 0.16824437583389304, 'drop_l3': 0.13803871786437744, 'drop_l4': 0.2153932351210886, 'drop_l5': 0.6263118179184833, 'drop_l6': 0.5527302347196649, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:18:19,273] Trial 631 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.1886058444893306, 'drop_l2': 0.5488563723510385, 'drop_l3': 0.1349763567840019, 'drop_l4': 0.2211077312918479, 'drop_l5': 0.3973279925648649, 'drop_l6': 0.5239026565553686, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:18:20,481] Trial 630 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.1417654589141532, 'drop_l2': 0.10026340478878387, 'drop_l3': 0.12391783480078, 'drop_l4': 0.2221027032589957, 'drop_l5': 0.6506445096517585, 'drop_l6': 0.5552714961014714, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:18:20,504] Trial 629 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.12971962239833687, 'drop_l2': 0.7641065731047612, 'drop_l3': 0.13078670149523164, 'drop_l4': 0.20938270105984633, 'drop_l5': 0.5973482606041599, 'drop_l6': 0.5012101274139874, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:18:36,528] Trial 632 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.37134431202217794, 'drop_l2': 0.21439307861457735, 'drop_l3': 0.13241667701269494, 'drop_l4': 0.22428210178603314, 'drop_l5': 0.6348505232996661, 'drop_l6': 0.5576235866292268, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:19:00,132] Trial 635 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.1279405599098375, 'drop_l2': 0.21133491676018915, 'drop_l3': 0.172829491736274, 'drop_l4': 0.5868306472131588, 'drop_l5': 0.6276796734038433, 'drop_l6': 0.5587056625317144, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:19:01,130] Trial 634 finished with value: 0.546875 and parameters: {'drop_l1': 0.11292162711935787, 'drop_l2': 0.6612954827458959, 'drop_l3': 0.17124638804611694, 'drop_l4': 0.19134098257859064, 'drop_l5': 0.6305233457568004, 'drop_l6': 0.5512646494647836, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:19:01,879] Trial 633 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.10094760408348617, 'drop_l2': 0.16968830572362287, 'drop_l3': 0.24406677377423003, 'drop_l4': 0.18653517706071462, 'drop_l5': 0.6282766991769761, 'drop_l6': 0.5575025926932032, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:19:09,768] Trial 636 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.10034321954598899, 'drop_l2': 0.19472266722751055, 'drop_l3': 0.16796572476747956, 'drop_l4': 0.18779354354768563, 'drop_l5': 0.6077405803857163, 'drop_l6': 0.5412379555809995, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:19:39,943] Trial 637 finished with value: 0.609375 and parameters: {'drop_l1': 0.16687551992155478, 'drop_l2': 0.16076707374867466, 'drop_l3': 0.14367997492275086, 'drop_l4': 0.1898532190560833, 'drop_l5': 0.6436994626747778, 'drop_l6': 0.5709606625278616, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:19:41,365] Trial 638 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.14427328174035625, 'drop_l2': 0.17116501671098763, 'drop_l3': 0.10019519329701548, 'drop_l4': 0.2675976776567124, 'drop_l5': 0.6659054412349943, 'drop_l6': 0.7320247933549112, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:19:43,025] Trial 639 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.7229241651289108, 'drop_l2': 0.5690307927127716, 'drop_l3': 0.1514575353467313, 'drop_l4': 0.1987764197950888, 'drop_l5': 0.6041810129060425, 'drop_l6': 0.7978493155303528, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:19:46,218] Trial 640 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.11441280374857651, 'drop_l2': 0.1644667543580396, 'drop_l3': 0.10191451258315741, 'drop_l4': 0.26595094842938904, 'drop_l5': 0.6458729044005365, 'drop_l6': 0.7865002512390501, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:20:22,434] Trial 641 finished with value: 0.6625000238418579 and parameters: {'drop_l1': 0.14539674046601725, 'drop_l2': 0.16445192183360394, 'drop_l3': 0.15218724470528122, 'drop_l4': 0.17060309948282074, 'drop_l5': 0.6093318165764657, 'drop_l6': 0.5804091578225141, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:20:24,288] Trial 643 finished with value: 0.65625 and parameters: {'drop_l1': 0.11368982405444356, 'drop_l2': 0.15830694808460655, 'drop_l3': 0.24057068148346594, 'drop_l4': 0.17888809998985133, 'drop_l5': 0.6134520138515706, 'drop_l6': 0.7914112408770827, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:20:24,291] Trial 642 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.11343455440368849, 'drop_l2': 0.5632116004216473, 'drop_l3': 0.2436214164988107, 'drop_l4': 0.17800226157870047, 'drop_l5': 0.5872503256366101, 'drop_l6': 0.8021806532542813, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:20:26,532] Trial 644 finished with value: 0.640625 and parameters: {'drop_l1': 0.10110819912087499, 'drop_l2': 0.15153228806559016, 'drop_l3': 0.19073825961259397, 'drop_l4': 0.23466676978719186, 'drop_l5': 0.6153997597799022, 'drop_l6': 0.5732606663878914, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:21:03,906] Trial 645 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.1437342343871616, 'drop_l2': 0.1870267550802357, 'drop_l3': 0.15069390152708423, 'drop_l4': 0.5202329860836992, 'drop_l5': 0.5844452728854569, 'drop_l6': 0.5720156584367371, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:21:05,425] Trial 646 finished with value: 0.671875 and parameters: {'drop_l1': 0.14475457106316017, 'drop_l2': 0.1840955704046862, 'drop_l3': 0.14563405338330845, 'drop_l4': 0.16396781083641282, 'drop_l5': 0.6178688683946675, 'drop_l6': 0.5790086638802115, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:21:07,558] Trial 648 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.1479780249190148, 'drop_l2': 0.18517298872267857, 'drop_l3': 0.15007546212643458, 'drop_l4': 0.20998610721835512, 'drop_l5': 0.592800619324513, 'drop_l6': 0.5762969166028578, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:21:07,962] Trial 647 finished with value: 0.625 and parameters: {'drop_l1': 0.14750542132895095, 'drop_l2': 0.18624482582790203, 'drop_l3': 0.14738079723138298, 'drop_l4': 0.22856969781412664, 'drop_l5': 0.6185155372980683, 'drop_l6': 0.574120215851878, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:21:43,716] Trial 649 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.1533747501420204, 'drop_l2': 0.17929132342868448, 'drop_l3': 0.12517810668888343, 'drop_l4': 0.17262266383609415, 'drop_l5': 0.6259238591636127, 'drop_l6': 0.5449069902680558, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:21:45,319] Trial 650 finished with value: 0.5531250238418579 and parameters: {'drop_l1': 0.13202937391216207, 'drop_l2': 0.18746622581423228, 'drop_l3': 0.1268387978854474, 'drop_l4': 0.2072032760732492, 'drop_l5': 0.617649176736909, 'drop_l6': 0.546037123301367, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:21:47,692] Trial 652 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.12991709745861113, 'drop_l2': 0.21415297105874534, 'drop_l3': 0.11696948803252294, 'drop_l4': 0.24435432397605353, 'drop_l5': 0.6326113161039176, 'drop_l6': 0.5575163270939049, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:21:47,806] Trial 651 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.16056339790956026, 'drop_l2': 0.18134066969366094, 'drop_l3': 0.12964805853770003, 'drop_l4': 0.16405931523522554, 'drop_l5': 0.6907036070019736, 'drop_l6': 0.5884629866262875, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:22:22,687] Trial 653 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.12914042635397852, 'drop_l2': 0.19641524755688913, 'drop_l3': 0.11831658546416993, 'drop_l4': 0.2504379312220577, 'drop_l5': 0.6498652252788182, 'drop_l6': 0.5539421543048035, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:22:24,328] Trial 654 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.1280839989977631, 'drop_l2': 0.2241822618141029, 'drop_l3': 0.11536995646317984, 'drop_l4': 0.23512257618768798, 'drop_l5': 0.6805382054889134, 'drop_l6': 0.5562710510035869, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:22:26,713] Trial 655 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.14012817420893478, 'drop_l2': 0.22560186159303713, 'drop_l3': 0.13456790167027013, 'drop_l4': 0.49338497514536184, 'drop_l5': 0.6249560609246814, 'drop_l6': 0.5904087271299785, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:22:26,896] Trial 656 finished with value: 0.5874999761581421 and parameters: {'drop_l1': 0.16880677193938934, 'drop_l2': 0.23464780001819505, 'drop_l3': 0.11629980824074516, 'drop_l4': 0.16401925866468417, 'drop_l5': 0.6485739142819559, 'drop_l6': 0.591547698489268, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:23:02,442] Trial 657 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.1276672543797038, 'drop_l2': 0.21589005234815084, 'drop_l3': 0.10671186299071468, 'drop_l4': 0.22783982189415147, 'drop_l5': 0.651213569307077, 'drop_l6': 0.559527448111929, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:23:06,083] Trial 658 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.16238708167258192, 'drop_l2': 0.16985053930049826, 'drop_l3': 0.11327827606150533, 'drop_l4': 0.16519510911346696, 'drop_l5': 0.657225013563683, 'drop_l6': 0.596647608809422, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:23:06,877] Trial 660 finished with value: 0.65625 and parameters: {'drop_l1': 0.1239114378904135, 'drop_l2': 0.18474882656625752, 'drop_l3': 0.11198919085661035, 'drop_l4': 0.25005199265421607, 'drop_l5': 0.6645854836165067, 'drop_l6': 0.534318791717055, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:23:07,055] Trial 659 finished with value: 0.625 and parameters: {'drop_l1': 0.15712358042086771, 'drop_l2': 0.226905781099225, 'drop_l3': 0.13904637651936352, 'drop_l4': 0.1591838170308754, 'drop_l5': 0.647155819612702, 'drop_l6': 0.5373535682055625, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:23:42,769] Trial 661 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.1400226598811499, 'drop_l2': 0.17122541989896126, 'drop_l3': 0.13541748614434612, 'drop_l4': 0.16687534554863512, 'drop_l5': 0.607191410327485, 'drop_l6': 0.5403124502549469, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:23:47,659] Trial 663 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.10124201054956772, 'drop_l2': 0.2586800383105194, 'drop_l3': 0.13339045279389106, 'drop_l4': 0.16961771853526492, 'drop_l5': 0.6243782065441931, 'drop_l6': 0.7579777437334657, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:23:48,812] Trial 662 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.13864509022376653, 'drop_l2': 0.20815203863939657, 'drop_l3': 0.13813403796322093, 'drop_l4': 0.16885589751085375, 'drop_l5': 0.6079462654805795, 'drop_l6': 0.5351160929524384, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:23:48,946] Trial 664 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.11604442111081271, 'drop_l2': 0.19920539186823238, 'drop_l3': 0.10173527159092632, 'drop_l4': 0.37820266490642285, 'drop_l5': 0.6387628122184988, 'drop_l6': 0.5222144201134018, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:24:17,674] Trial 665 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.134383199421838, 'drop_l2': 0.216237502734807, 'drop_l3': 0.20446679415297458, 'drop_l4': 0.2849114718110816, 'drop_l5': 0.6406986890146024, 'drop_l6': 0.7512295327010233, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:24:30,742] Trial 666 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.11828424346358615, 'drop_l2': 0.19090623529205356, 'drop_l3': 0.30509976608887857, 'drop_l4': 0.25526997307365257, 'drop_l5': 0.63993634984452, 'drop_l6': 0.5635727046105391, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:24:31,222] Trial 667 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.11410508325021854, 'drop_l2': 0.2066879022530184, 'drop_l3': 0.10001187522470956, 'drop_l4': 0.27459734590611257, 'drop_l5': 0.5862391659634051, 'drop_l6': 0.5673422147453401, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:24:32,226] Trial 668 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.1254952454890888, 'drop_l2': 0.20326020893183347, 'drop_l3': 0.41283229211447187, 'drop_l4': 0.27444386057968606, 'drop_l5': 0.6295095474556204, 'drop_l6': 0.7499139820778642, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:24:48,833] Trial 669 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.10018134661667237, 'drop_l2': 0.15394270280585595, 'drop_l3': 0.1625108406720004, 'drop_l4': 0.2624761233178188, 'drop_l5': 0.5926157440189216, 'drop_l6': 0.77411304034865, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:25:12,013] Trial 670 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.16261099276547336, 'drop_l2': 0.19616847295670697, 'drop_l3': 0.16561188294070486, 'drop_l4': 0.26956822247024226, 'drop_l5': 0.5940814186669746, 'drop_l6': 0.7757276282959713, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:25:13,463] Trial 671 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.15837214544384046, 'drop_l2': 0.16337960571404558, 'drop_l3': 0.16113370926797674, 'drop_l4': 0.19047082690659856, 'drop_l5': 0.5979438613935907, 'drop_l6': 0.7416910299336865, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:25:13,671] Trial 672 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.15909297099640674, 'drop_l2': 0.20259134658982914, 'drop_l3': 0.15861004800425063, 'drop_l4': 0.3012821317879865, 'drop_l5': 0.601793690231903, 'drop_l6': 0.5831133160248712, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:25:22,395] Trial 673 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.15567830469926616, 'drop_l2': 0.16409964266339727, 'drop_l3': 0.1569584494696644, 'drop_l4': 0.19233721793092662, 'drop_l5': 0.6225236974358006, 'drop_l6': 0.5789388355697287, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:25:54,512] Trial 676 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.14419981277197924, 'drop_l2': 0.17471196100725217, 'drop_l3': 0.1854425258290025, 'drop_l4': 0.2463080048451636, 'drop_l5': 0.61168334686, 'drop_l6': 0.6025522177969834, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:25:55,476] Trial 674 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.15070629219192153, 'drop_l2': 0.19749758895678307, 'drop_l3': 0.18014055138352292, 'drop_l4': 0.2407409607515371, 'drop_l5': 0.5693526363389997, 'drop_l6': 0.7308865425943227, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:25:55,485] Trial 675 finished with value: 0.5874999761581421 and parameters: {'drop_l1': 0.14127977700159466, 'drop_l2': 0.20378464339623306, 'drop_l3': 0.18709093912807317, 'drop_l4': 0.2431719942527512, 'drop_l5': 0.669617638424061, 'drop_l6': 0.5763096079046901, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:25:58,710] Trial 677 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.13278028310784484, 'drop_l2': 0.1394883896070593, 'drop_l3': 0.11791177552025969, 'drop_l4': 0.21294724078184157, 'drop_l5': 0.6708032746783336, 'drop_l6': 0.46425375002369007, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:26:35,727] Trial 678 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.39902833972351814, 'drop_l2': 0.14203519233714976, 'drop_l3': 0.125688363294179, 'drop_l4': 0.21478270796155094, 'drop_l5': 0.5596375182376863, 'drop_l6': 0.72823310386449, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:26:38,493] Trial 680 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.10053748983779899, 'drop_l2': 0.1450689256838235, 'drop_l3': 0.12483140977039658, 'drop_l4': 0.20602183710195965, 'drop_l5': 0.6059688595362267, 'drop_l6': 0.5523791677926629, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:26:39,036] Trial 681 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.10102285960773483, 'drop_l2': 0.13746019953288793, 'drop_l3': 0.10049132546738297, 'drop_l4': 0.2198557405874473, 'drop_l5': 0.31258854210805215, 'drop_l6': 0.5072493985623654, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:26:39,262] Trial 679 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.17292854178360476, 'drop_l2': 0.14694009977855643, 'drop_l3': 0.12108643675302963, 'drop_l4': 0.2035087465737085, 'drop_l5': 0.3039418403754696, 'drop_l6': 0.5084616991401234, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:27:17,013] Trial 682 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.1759010591856661, 'drop_l2': 0.17643308449503392, 'drop_l3': 0.149538763866081, 'drop_l4': 0.19939624876257048, 'drop_l5': 0.6271614330196901, 'drop_l6': 0.6003233408711516, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:27:21,778] Trial 683 finished with value: 0.640625 and parameters: {'drop_l1': 0.1737577444252374, 'drop_l2': 0.16024756719727232, 'drop_l3': 0.14627398472718672, 'drop_l4': 0.18494878677465226, 'drop_l5': 0.5775994294454119, 'drop_l6': 0.5846677502966352, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:27:22,098] Trial 685 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.11205794134606187, 'drop_l2': 0.1564658411448961, 'drop_l3': 0.14465285190936195, 'drop_l4': 0.18046123705252767, 'drop_l5': 0.580486568577959, 'drop_l6': 0.5685229310778974, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:27:22,181] Trial 684 finished with value: 0.5843750238418579 and parameters: {'drop_l1': 0.7700350225838128, 'drop_l2': 0.12336868480123439, 'drop_l3': 0.11225597714264167, 'drop_l4': 0.2242372937686362, 'drop_l5': 0.5851129091339179, 'drop_l6': 0.5734785528130789, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:27:52,838] Trial 686 finished with value: 0.640625 and parameters: {'drop_l1': 0.12965131901780613, 'drop_l2': 0.24220284954603527, 'drop_l3': 0.19878920166824982, 'drop_l4': 0.2341904250342999, 'drop_l5': 0.32462197216131344, 'drop_l6': 0.7550671675493391, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:28:00,986] Trial 688 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.1433185045547623, 'drop_l2': 0.13467484947890246, 'drop_l3': 0.14511037268331606, 'drop_l4': 0.2577868910292656, 'drop_l5': 0.6359988262132648, 'drop_l6': 0.6097315096381608, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:28:02,632] Trial 687 finished with value: 0.65625 and parameters: {'drop_l1': 0.10029652905520361, 'drop_l2': 0.1265025645197998, 'drop_l3': 0.11344444052542911, 'drop_l4': 0.22519072267704354, 'drop_l5': 0.5791145493572015, 'drop_l6': 0.554717378280427, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:28:02,648] Trial 689 finished with value: 0.609375 and parameters: {'drop_l1': 0.14375687425921194, 'drop_l2': 0.24001999968486365, 'drop_l3': 0.14418860651595244, 'drop_l4': 0.25227869607521985, 'drop_l5': 0.6144577420494908, 'drop_l6': 0.6101115638305292, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:28:22,270] Trial 690 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.10083746169533031, 'drop_l2': 0.12948722988268913, 'drop_l3': 0.211642397285954, 'drop_l4': 0.22864266712001952, 'drop_l5': 0.3021630455209694, 'drop_l6': 0.5491141251950826, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:28:40,542] Trial 691 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.13771907064083277, 'drop_l2': 0.5905296546889595, 'drop_l3': 0.15816759834213998, 'drop_l4': 0.1907581298717013, 'drop_l5': 0.6127225415166583, 'drop_l6': 0.560930670231222, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:28:42,373] Trial 693 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.12345510689913444, 'drop_l2': 0.1692128097819017, 'drop_l3': 0.1299127057694302, 'drop_l4': 0.18174650128986475, 'drop_l5': 0.2921169455091287, 'drop_l6': 0.5510941823046663, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:28:42,814] Trial 692 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.12172366852031673, 'drop_l2': 0.17168867072276767, 'drop_l3': 0.1289398817726225, 'drop_l4': 0.17675590986824352, 'drop_l5': 0.30295501683481496, 'drop_l6': 0.5443689708645113, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:28:54,992] Trial 694 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.12594444744902394, 'drop_l2': 0.5902778729923244, 'drop_l3': 0.13086940737398609, 'drop_l4': 0.14379020401183676, 'drop_l5': 0.6187443139072906, 'drop_l6': 0.7152256144095612, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:29:19,403] Trial 695 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.12575035452596106, 'drop_l2': 0.2161676527548973, 'drop_l3': 0.17298389966344058, 'drop_l4': 0.15872760024763347, 'drop_l5': 0.6319395353227794, 'drop_l6': 0.5875195810780722, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:29:23,110] Trial 697 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.15059398643716304, 'drop_l2': 0.16165137359360002, 'drop_l3': 0.16011768706233023, 'drop_l4': 0.1537591035969166, 'drop_l5': 0.6021346616750369, 'drop_l6': 0.5806168909049431, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:29:23,484] Trial 696 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.11325219279095367, 'drop_l2': 0.15353192316488914, 'drop_l3': 0.10053092301533544, 'drop_l4': 0.16433475244575357, 'drop_l5': 0.5970482144679478, 'drop_l6': 0.5838196946011728, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:29:29,970] Trial 698 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.15369771835287746, 'drop_l2': 0.17040179269777106, 'drop_l3': 0.17404963152106695, 'drop_l4': 0.1784929778400045, 'drop_l5': 0.2947023979446196, 'drop_l6': 0.7759964139560607, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:29:56,124] Trial 699 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.11247395728192174, 'drop_l2': 0.15636532008138718, 'drop_l3': 0.10095806820396834, 'drop_l4': 0.20078260864111072, 'drop_l5': 0.5566179059021815, 'drop_l6': 0.5248205890795538, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:30:04,795] Trial 700 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.11255228813172985, 'drop_l2': 0.1759683479832722, 'drop_l3': 0.13381092306027192, 'drop_l4': 0.17565328539232955, 'drop_l5': 0.6017594488508061, 'drop_l6': 0.5737196084722166, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:30:05,116] Trial 701 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.2511571969741661, 'drop_l2': 0.152957746069786, 'drop_l3': 0.1004131404467736, 'drop_l4': 0.17478203307807633, 'drop_l5': 0.555606995062869, 'drop_l6': 0.8281632373718792, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:30:07,905] Trial 702 finished with value: 0.668749988079071 and parameters: {'drop_l1': 0.11814002864772286, 'drop_l2': 0.15227179371506974, 'drop_l3': 0.13334432799280213, 'drop_l4': 0.19038248376986802, 'drop_l5': 0.5700470056889139, 'drop_l6': 0.5678925912727634, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:30:27,252] Trial 703 finished with value: 0.609375 and parameters: {'drop_l1': 0.1627885982729707, 'drop_l2': 0.19032363330060925, 'drop_l3': 0.14592005638262467, 'drop_l4': 0.1772260524490166, 'drop_l5': 0.35105205338090834, 'drop_l6': 0.8429443961040142, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:30:45,367] Trial 704 finished with value: 0.6625000238418579 and parameters: {'drop_l1': 0.16608221869268247, 'drop_l2': 0.1446337169394184, 'drop_l3': 0.4450593519507605, 'drop_l4': 0.19009879621346104, 'drop_l5': 0.2845035759149095, 'drop_l6': 0.1015115788527759, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:30:45,927] Trial 705 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.1003810034742397, 'drop_l2': 0.18177693447861548, 'drop_l3': 0.10104460710166092, 'drop_l4': 0.2371692826169453, 'drop_l5': 0.338362485988234, 'drop_l6': 0.5306596033095237, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:30:47,420] Trial 706 finished with value: 0.609375 and parameters: {'drop_l1': 0.12829545872021822, 'drop_l2': 0.17563106645580623, 'drop_l3': 0.13333211858992458, 'drop_l4': 0.19201359323350708, 'drop_l5': 0.5684942418621954, 'drop_l6': 0.7603314911219065, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:31:01,452] Trial 707 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.10065673218099411, 'drop_l2': 0.13888597779855083, 'drop_l3': 0.12951879610108954, 'drop_l4': 0.19724843819335708, 'drop_l5': 0.2834503117803829, 'drop_l6': 0.5593284153950003, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:31:24,303] Trial 708 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.10013498650686367, 'drop_l2': 0.16559461964569136, 'drop_l3': 0.1179079703645748, 'drop_l4': 0.14834727185011584, 'drop_l5': 0.5793866741426734, 'drop_l6': 0.5594570148759878, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:31:24,567] Trial 709 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.18845862975206784, 'drop_l2': 0.14353532328938376, 'drop_l3': 0.12505390478257053, 'drop_l4': 0.1996710759190528, 'drop_l5': 0.28172418744454353, 'drop_l6': 0.1158404468676506, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:31:25,911] Trial 710 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.12779960341806407, 'drop_l2': 0.1478734707392032, 'drop_l3': 0.29190094806765815, 'drop_l4': 0.20260652831321696, 'drop_l5': 0.5659818573194176, 'drop_l6': 0.5661827570378044, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:31:34,671] Trial 711 finished with value: 0.609375 and parameters: {'drop_l1': 0.10203607224620871, 'drop_l2': 0.1285608694355048, 'drop_l3': 0.11431712979853116, 'drop_l4': 0.21228411584226817, 'drop_l5': 0.5695137418728804, 'drop_l6': 0.5179712418588901, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:32:02,337] Trial 712 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.12920744179258117, 'drop_l2': 0.19301082592492377, 'drop_l3': 0.4973389452949422, 'drop_l4': 0.2578165676397563, 'drop_l5': 0.5591154580141755, 'drop_l6': 0.7395525373973921, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:32:02,971] Trial 713 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.13954504988755478, 'drop_l2': 0.15771440636654166, 'drop_l3': 0.1427531819924866, 'drop_l4': 0.1548960365107539, 'drop_l5': 0.5446780346300228, 'drop_l6': 0.6052447362763335, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:32:03,451] Trial 714 finished with value: 0.640625 and parameters: {'drop_l1': 0.12688244347407873, 'drop_l2': 0.18404031127698683, 'drop_l3': 0.11692140335517028, 'drop_l4': 0.2148198685657547, 'drop_l5': 0.5453480164903208, 'drop_l6': 0.7383727172932274, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:32:07,461] Trial 715 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.13533803062997507, 'drop_l2': 0.18423958849391614, 'drop_l3': 0.33212948973784173, 'drop_l4': 0.19263185259706178, 'drop_l5': 0.5410540552298617, 'drop_l6': 0.7420148266688197, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:32:40,709] Trial 716 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.1136179440420057, 'drop_l2': 0.14902609987590698, 'drop_l3': 0.331412866146862, 'drop_l4': 0.20932054736600256, 'drop_l5': 0.5468587302202355, 'drop_l6': 0.5405953534222576, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:32:42,986] Trial 717 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.10003072516550889, 'drop_l2': 0.14930742251873097, 'drop_l3': 0.3490997660825388, 'drop_l4': 0.2137383197366634, 'drop_l5': 0.5648030359674994, 'drop_l6': 0.8135658482625303, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:32:43,066] Trial 718 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.14278287810632306, 'drop_l2': 0.15898521581943637, 'drop_l3': 0.3499907207310292, 'drop_l4': 0.1613032282023763, 'drop_l5': 0.5669255057543565, 'drop_l6': 0.6000275560477544, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:32:44,712] Trial 719 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.10030377674833567, 'drop_l2': 0.15818959755814982, 'drop_l3': 0.35462095952912476, 'drop_l4': 0.16491860924081747, 'drop_l5': 0.5759507468584751, 'drop_l6': 0.5930034155943089, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:33:21,758] Trial 720 finished with value: 0.578125 and parameters: {'drop_l1': 0.1006275137538276, 'drop_l2': 0.14990633375663248, 'drop_l3': 0.4092262312325348, 'drop_l4': 0.18487049165644742, 'drop_l5': 0.5696768909601969, 'drop_l6': 0.5677831411982177, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:33:22,966] Trial 721 finished with value: 0.65625 and parameters: {'drop_l1': 0.11575330830845779, 'drop_l2': 0.1470367229115057, 'drop_l3': 0.47782772897454673, 'drop_l4': 0.18852456358725894, 'drop_l5': 0.5587547053104356, 'drop_l6': 0.5686648649833369, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:33:23,605] Trial 722 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.11514627793412922, 'drop_l2': 0.13137095526807782, 'drop_l3': 0.11265014672768421, 'drop_l4': 0.1942349295636392, 'drop_l5': 0.5777751824882245, 'drop_l6': 0.5669615449599619, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:33:24,250] Trial 723 finished with value: 0.609375 and parameters: {'drop_l1': 0.144027906281391, 'drop_l2': 0.17257871812485578, 'drop_l3': 0.15325348082967435, 'drop_l4': 0.17564756934912049, 'drop_l5': 0.5951220761088684, 'drop_l6': 0.5732362715923351, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:34:02,812] Trial 724 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.15262170091080576, 'drop_l2': 0.1646422282179517, 'drop_l3': 0.1495429420445801, 'drop_l4': 0.1434305196534269, 'drop_l5': 0.5958404521620536, 'drop_l6': 0.5897140723275953, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:34:04,775] Trial 726 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.14439589818844334, 'drop_l2': 0.16498903291379524, 'drop_l3': 0.15382298285720145, 'drop_l4': 0.17316140747726497, 'drop_l5': 0.5910802975680257, 'drop_l6': 0.5910829044007936, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:34:04,995] Trial 727 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.12771283894127955, 'drop_l2': 0.17707238283407428, 'drop_l3': 0.1720951621350192, 'drop_l4': 0.22968128300321397, 'drop_l5': 0.5349189485367924, 'drop_l6': 0.7809626555483467, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:34:05,106] Trial 725 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.1764998373217668, 'drop_l2': 0.13612119929269303, 'drop_l3': 0.5929474186054079, 'drop_l4': 0.18685601732790424, 'drop_l5': 0.584313249060409, 'drop_l6': 0.779306123719162, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:34:41,021] Trial 728 finished with value: 0.515625 and parameters: {'drop_l1': 0.16957511606324233, 'drop_l2': 0.14044658783161246, 'drop_l3': 0.45118787443514846, 'drop_l4': 0.19098669391792739, 'drop_l5': 0.2657723650430776, 'drop_l6': 0.1070776799680202, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:34:43,927] Trial 729 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.12429952410590975, 'drop_l2': 0.17900062325479726, 'drop_l3': 0.29988343908210047, 'drop_l4': 0.15373262431031795, 'drop_l5': 0.6214530593693022, 'drop_l6': 0.6053712717647114, 'activation': 'tanh', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:34:44,411] Trial 730 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.16499030231666792, 'drop_l2': 0.1409351892245754, 'drop_l3': 0.28698989759730775, 'drop_l4': 0.15662969660733506, 'drop_l5': 0.6518720265647059, 'drop_l6': 0.620170372611575, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:34:44,465] Trial 731 finished with value: 0.534375011920929 and parameters: {'drop_l1': 0.1606299265624129, 'drop_l2': 0.1661655084299336, 'drop_l3': 0.4433609653468799, 'drop_l4': 0.15724123916513447, 'drop_l5': 0.6216477282948042, 'drop_l6': 0.6085229539999009, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:35:21,136] Trial 732 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.11317298072350515, 'drop_l2': 0.12604392376359064, 'drop_l3': 0.10099325432397999, 'drop_l4': 0.2097618541028478, 'drop_l5': 0.217601522776591, 'drop_l6': 0.5472363172556564, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:35:24,696] Trial 733 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.1197634838112473, 'drop_l2': 0.1289350903453939, 'drop_l3': 0.2913982738428456, 'drop_l4': 0.20730889110651818, 'drop_l5': 0.6556816388060895, 'drop_l6': 0.5522243345680157, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:35:24,732] Trial 734 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.11555465660552124, 'drop_l2': 0.5381843812881538, 'drop_l3': 0.10019934367206283, 'drop_l4': 0.24376873192463822, 'drop_l5': 0.2251708554964186, 'drop_l6': 0.7147637672467498, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:35:24,785] Trial 735 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.1000647083576206, 'drop_l2': 0.1291526367987615, 'drop_l3': 0.10002540216797577, 'drop_l4': 0.24263684346400574, 'drop_l5': 0.23088181763989285, 'drop_l6': 0.7556112201892915, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:35:58,793] Trial 736 finished with value: 0.621874988079071 and parameters: {'drop_l1': 0.1345425145870767, 'drop_l2': 0.5375005547987561, 'drop_l3': 0.13529893410564614, 'drop_l4': 0.17193419401454596, 'drop_l5': 0.6055778616162342, 'drop_l6': 0.5753540557000969, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:36:04,513] Trial 738 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.5714782630818757, 'drop_l2': 0.19706442219723136, 'drop_l3': 0.13250163712514587, 'drop_l4': 0.20428248776062477, 'drop_l5': 0.46541319452755747, 'drop_l6': 0.567258508611178, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:36:05,199] Trial 739 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.3360548968777092, 'drop_l2': 0.11088992080243851, 'drop_l3': 0.1367033222357223, 'drop_l4': 0.2035737288547273, 'drop_l5': 0.2951114075602206, 'drop_l6': 0.7871313284244382, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:36:05,204] Trial 737 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.13481150434970693, 'drop_l2': 0.1901807082675599, 'drop_l3': 0.13166911073916265, 'drop_l4': 0.201325984062021, 'drop_l5': 0.2971495638189908, 'drop_l6': 0.8035343013520402, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:36:32,050] Trial 740 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.13353563666872154, 'drop_l2': 0.1969637297613745, 'drop_l3': 0.13365311676059402, 'drop_l4': 0.29106309772650474, 'drop_l5': 0.26627906914051197, 'drop_l6': 0.7094560614613976, 'activation': 'relu', 'kernel_init': 'lecun_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:36:44,784] Trial 741 finished with value: 0.659375011920929 and parameters: {'drop_l1': 0.11354130969032723, 'drop_l2': 0.1475693547867169, 'drop_l3': 0.5229130079316682, 'drop_l4': 0.19248311638171337, 'drop_l5': 0.24353336538777443, 'drop_l6': 0.8085028503852281, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:36:45,552] Trial 743 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.12070661110557446, 'drop_l2': 0.20787826979788135, 'drop_l3': 0.2708698802488447, 'drop_l4': 0.2785739720846882, 'drop_l5': 0.3551410783731159, 'drop_l6': 0.714944168321376, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:36:46,573] Trial 742 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.12279767784978723, 'drop_l2': 0.22550711536463458, 'drop_l3': 0.27134614858941425, 'drop_l4': 0.22746700124676092, 'drop_l5': 0.5297580637301883, 'drop_l6': 0.7082158708038755, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:37:03,621] Trial 744 finished with value: 0.609375 and parameters: {'drop_l1': 0.1557735954466065, 'drop_l2': 0.1546125105620747, 'drop_l3': 0.14883459906913046, 'drop_l4': 0.17371229241430944, 'drop_l5': 0.5879899762083861, 'drop_l6': 0.5880064429254597, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:37:27,193] Trial 745 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.1495094328750738, 'drop_l2': 0.15479598166763534, 'drop_l3': 0.15414187800038961, 'drop_l4': 0.17413973032730937, 'drop_l5': 0.6358813996593607, 'drop_l6': 0.5882333889085531, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:37:28,211] Trial 747 finished with value: 0.625 and parameters: {'drop_l1': 0.5259992011383471, 'drop_l2': 0.11458297194765965, 'drop_l3': 0.2786038389240735, 'drop_l4': 0.4369009360113379, 'drop_l5': 0.5609607373627861, 'drop_l6': 0.55552779341734, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:37:28,212] Trial 746 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.12712196394034206, 'drop_l2': 0.10024947742599867, 'drop_l3': 0.19211778349734246, 'drop_l4': 0.4377056135134202, 'drop_l5': 0.5506970832295622, 'drop_l6': 0.7534720643747987, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:37:37,815] Trial 748 finished with value: 0.640625 and parameters: {'drop_l1': 0.10049256668877334, 'drop_l2': 0.10008969942332754, 'drop_l3': 0.3212075066585214, 'drop_l4': 0.21927921724575658, 'drop_l5': 0.5551354189356398, 'drop_l6': 0.5528527238718393, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:38:09,346] Trial 749 finished with value: 0.640625 and parameters: {'drop_l1': 0.10006553051726956, 'drop_l2': 0.10187260100881108, 'drop_l3': 0.2825737646181519, 'drop_l4': 0.22078343566365827, 'drop_l5': 0.5568482092026918, 'drop_l6': 0.5536935503427828, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:38:09,821] Trial 751 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.11233004408382438, 'drop_l2': 0.1345111466808818, 'drop_l3': 0.11636717706839636, 'drop_l4': 0.22282451599848627, 'drop_l5': 0.32457027436581526, 'drop_l6': 0.7667718992938428, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:38:10,343] Trial 750 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.10006788760136899, 'drop_l2': 0.13370344631402672, 'drop_l3': 0.3171817082941158, 'drop_l4': 0.40675611218693575, 'drop_l5': 0.331107689174389, 'drop_l6': 0.7440279288292254, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:38:15,508] Trial 752 finished with value: 0.5843750238418579 and parameters: {'drop_l1': 0.13970344225014336, 'drop_l2': 0.13607963296221304, 'drop_l3': 0.1716993839587909, 'drop_l4': 0.3364683514246194, 'drop_l5': 0.6079865283099561, 'drop_l6': 0.578638592031207, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:38:49,943] Trial 753 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.10031797257594116, 'drop_l2': 0.13480244220600732, 'drop_l3': 0.11513828081076062, 'drop_l4': 0.3373585625010564, 'drop_l5': 0.6329665952070997, 'drop_l6': 0.5741570828975291, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:38:50,913] Trial 755 finished with value: 0.59375 and parameters: {'drop_l1': 0.20060854646756393, 'drop_l2': 0.14549600354449382, 'drop_l3': 0.16497850566030114, 'drop_l4': 0.1910482739845509, 'drop_l5': 0.5780493429307225, 'drop_l6': 0.14140112988606482, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:38:50,925] Trial 754 finished with value: 0.596875011920929 and parameters: {'drop_l1': 0.11412027944863926, 'drop_l2': 0.17844401497817608, 'drop_l3': 0.11473750287404073, 'drop_l4': 0.2590467496517469, 'drop_l5': 0.21006747074754112, 'drop_l6': 0.7293675630344952, 'activation': 'relu', 'kernel_init': 'normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:38:52,449] Trial 756 finished with value: 0.612500011920929 and parameters: {'drop_l1': 0.14412228304595295, 'drop_l2': 0.16908863374319852, 'drop_l3': 0.16262799605748332, 'drop_l4': 0.18775981832028976, 'drop_l5': 0.5822681422124647, 'drop_l6': 0.5801092518872072, 'activation': 'relu', 'kernel_init': 'glorot_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:39:30,489] Trial 757 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.12367434608281395, 'drop_l2': 0.11576259986658044, 'drop_l3': 0.2913875588232472, 'drop_l4': 0.23268788758677364, 'drop_l5': 0.5776094589278796, 'drop_l6': 0.5327128525871834, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:39:32,341] Trial 760 finished with value: 0.6000000238418579 and parameters: {'drop_l1': 0.10099320017416874, 'drop_l2': 0.12425112749166044, 'drop_l3': 0.11435698719015214, 'drop_l4': 0.22167449380062682, 'drop_l5': 0.2556139703069391, 'drop_l6': 0.5420591063998421, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:39:32,402] Trial 759 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.1276104905938234, 'drop_l2': 0.1835568522753629, 'drop_l3': 0.2853479683691933, 'drop_l4': 0.24685628490966785, 'drop_l5': 0.3700339973417447, 'drop_l6': 0.7726796837575084, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:39:33,232] Trial 758 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.11482554360042901, 'drop_l2': 0.22826437953423112, 'drop_l3': 0.2922117943718175, 'drop_l4': 0.24582887185324342, 'drop_l5': 0.35912902856225903, 'drop_l6': 0.7319870731900721, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:40:12,078] Trial 761 finished with value: 0.625 and parameters: {'drop_l1': 0.13626301242055694, 'drop_l2': 0.2561846894855897, 'drop_l3': 0.30108740504871456, 'drop_l4': 0.23408334041658863, 'drop_l5': 0.38699531671538273, 'drop_l6': 0.7917501127549642, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:40:15,943] Trial 763 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.15710341033948738, 'drop_l2': 0.10005724250257506, 'drop_l3': 0.3810050573898221, 'drop_l4': 0.19463967598081233, 'drop_l5': 0.2888881256039227, 'drop_l6': 0.5670741899417353, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:40:15,954] Trial 764 finished with value: 0.65625 and parameters: {'drop_l1': 0.15428965154582872, 'drop_l2': 0.1124216787313204, 'drop_l3': 0.12753079860511105, 'drop_l4': 0.2063038030681824, 'drop_l5': 0.31861302805198277, 'drop_l6': 0.7901549158708594, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:40:15,991] Trial 762 finished with value: 0.625 and parameters: {'drop_l1': 0.15289882827981793, 'drop_l2': 0.10014993232647966, 'drop_l3': 0.7200114280157346, 'drop_l4': 0.20036211915509447, 'drop_l5': 0.28878529847792217, 'drop_l6': 0.7942235782691512, 'activation': 'relu', 'kernel_init': 'he_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:40:52,526] Trial 765 finished with value: 0.675000011920929 and parameters: {'drop_l1': 0.150284228015943, 'drop_l2': 0.11812076489714503, 'drop_l3': 0.26604404710381324, 'drop_l4': 0.21185993146413848, 'drop_l5': 0.4021951252339352, 'drop_l6': 0.5972527278340845, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:40:57,320] Trial 768 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.10114908169311823, 'drop_l2': 0.12107854554398619, 'drop_l3': 0.19980974930934256, 'drop_l4': 0.2169040750785503, 'drop_l5': 0.19955583307729116, 'drop_l6': 0.5274614503346616, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:40:58,304] Trial 766 finished with value: 0.625 and parameters: {'drop_l1': 0.1821045976656621, 'drop_l2': 0.11515589264035606, 'drop_l3': 0.10218844002562999, 'drop_l4': 0.21491246531732736, 'drop_l5': 0.20968332601301445, 'drop_l6': 0.5270965699855582, 'activation': 'relu', 'kernel_init': 'uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:40:58,321] Trial 767 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.11391398557300267, 'drop_l2': 0.10040318616631727, 'drop_l3': 0.20828426688237348, 'drop_l4': 0.21573751690648563, 'drop_l5': 0.20377061883473502, 'drop_l6': 0.5314490751495276, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:41:28,445] Trial 769 finished with value: 0.6656249761581421 and parameters: {'drop_l1': 0.11358183504742726, 'drop_l2': 0.11365883868261835, 'drop_l3': 0.11327628407104366, 'drop_l4': 0.2117837563619796, 'drop_l5': 0.5930161527046284, 'drop_l6': 0.622371107880422, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:41:39,415] Trial 771 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.18964302271563113, 'drop_l2': 0.1521073233194229, 'drop_l3': 0.2587102268460558, 'drop_l4': 0.2306315837782478, 'drop_l5': 0.40311343327120147, 'drop_l6': 0.6096358580975073, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:41:40,700] Trial 770 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.1820796842569451, 'drop_l2': 0.11576956775356392, 'drop_l3': 0.799342047320089, 'drop_l4': 0.21403948266933376, 'drop_l5': 0.4071531322778318, 'drop_l6': 0.6009267718376351, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:41:40,701] Trial 772 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.13737036420203938, 'drop_l2': 0.15343302020920857, 'drop_l3': 0.24317759995503357, 'drop_l4': 0.23917917511745415, 'drop_l5': 0.39479967473062055, 'drop_l6': 0.6029401845206409, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:41:57,435] Trial 773 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.13797255894810406, 'drop_l2': 0.12257454335391653, 'drop_l3': 0.12478660576311834, 'drop_l4': 0.18407522637172577, 'drop_l5': 0.595828807912917, 'drop_l6': 0.6208250654373436, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:42:18,984] Trial 774 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.13781277029645417, 'drop_l2': 0.12839885680071583, 'drop_l3': 0.1267240397723219, 'drop_l4': 0.199661538194495, 'drop_l5': 0.5952572795696331, 'drop_l6': 0.6191404264889254, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:42:20,313] Trial 776 finished with value: 0.6468750238418579 and parameters: {'drop_l1': 0.13119108878264554, 'drop_l2': 0.12463962936289717, 'drop_l3': 0.12413682747142925, 'drop_l4': 0.18569967246546762, 'drop_l5': 0.5903031321581611, 'drop_l6': 0.6192960401505958, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:42:20,798] Trial 775 finished with value: 0.6499999761581421 and parameters: {'drop_l1': 0.1318795334994622, 'drop_l2': 0.1285811670074519, 'drop_l3': 0.12476370446684003, 'drop_l4': 0.18663323758205178, 'drop_l5': 0.5985271446255084, 'drop_l6': 0.6222896046935478, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:42:29,804] Trial 777 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.4661825528998276, 'drop_l2': 0.13931334844582358, 'drop_l3': 0.3098542704810644, 'drop_l4': 0.19995442610360414, 'drop_l5': 0.5688887020125809, 'drop_l6': 0.6177653415115016, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:42:59,950] Trial 780 finished with value: 0.559374988079071 and parameters: {'drop_l1': 0.11759822678369267, 'drop_l2': 0.1003678417199019, 'drop_l3': 0.2678451887615025, 'drop_l4': 0.20147905657989912, 'drop_l5': 0.6219047570341968, 'drop_l6': 0.6261368449987605, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:43:00,775] Trial 779 finished with value: 0.625 and parameters: {'drop_l1': 0.11614218796367613, 'drop_l2': 0.10038883795512571, 'drop_l3': 0.10162117512694141, 'drop_l4': 0.20479022882896245, 'drop_l5': 0.6140895746153973, 'drop_l6': 0.6120680838382483, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:43:01,019] Trial 778 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.1252475234056745, 'drop_l2': 0.13629385283749115, 'drop_l3': 0.10132700967642311, 'drop_l4': 0.20242550733530396, 'drop_l5': 0.6158031358915081, 'drop_l6': 0.6188010597239074, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:43:05,539] Trial 781 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.11549765402121107, 'drop_l2': 0.1449742275029864, 'drop_l3': 0.1451459820413714, 'drop_l4': 0.22556568385522843, 'drop_l5': 0.5349917307156371, 'drop_l6': 0.5983880227609499, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:43:44,206] Trial 783 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.14730497276906646, 'drop_l2': 0.14557616324594436, 'drop_l3': 0.1389557161205951, 'drop_l4': 0.23032991210507453, 'drop_l5': 0.5322963777207134, 'drop_l6': 0.6367144870169961, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:43:44,215] Trial 784 finished with value: 0.6343749761581421 and parameters: {'drop_l1': 0.10104929478760744, 'drop_l2': 0.10029471424799077, 'drop_l3': 0.31139087934209186, 'drop_l4': 0.2282516044884464, 'drop_l5': 0.5297237237708216, 'drop_l6': 0.5962719813911802, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:43:44,235] Trial 782 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.11395864897527336, 'drop_l2': 0.14094514038749484, 'drop_l3': 0.1381871115542214, 'drop_l4': 0.2216549719073093, 'drop_l5': 0.6383393046137468, 'drop_l6': 0.5978899338610413, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:43:45,692] Trial 785 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.14849300280061492, 'drop_l2': 0.11690483335644238, 'drop_l3': 0.13778374708780394, 'drop_l4': 0.2351833754373053, 'drop_l5': 0.6601852315120581, 'drop_l6': 0.6341213962677861, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:44:27,030] Trial 786 finished with value: 0.590624988079071 and parameters: {'drop_l1': 0.63731834938848, 'drop_l2': 0.11487969289673937, 'drop_l3': 0.1137218006827892, 'drop_l4': 0.17558990530867746, 'drop_l5': 0.5680616896092047, 'drop_l6': 0.5920935985587683, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:44:28,930] Trial 789 finished with value: 0.606249988079071 and parameters: {'drop_l1': 0.6426775334943131, 'drop_l2': 0.15874431567956812, 'drop_l3': 0.6181564141656065, 'drop_l4': 0.17651623547330653, 'drop_l5': 0.5599702987549628, 'drop_l6': 0.5904861459127586, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:44:29,533] Trial 788 finished with value: 0.637499988079071 and parameters: {'drop_l1': 0.1465202284823115, 'drop_l2': 0.16497360306356765, 'drop_l3': 0.10104788862027081, 'drop_l4': 0.1781255352560078, 'drop_l5': 0.6604878543908527, 'drop_l6': 0.5913184977609295, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:44:29,560] Trial 787 finished with value: 0.6031249761581421 and parameters: {'drop_l1': 0.8925340179112786, 'drop_l2': 0.11522104435155534, 'drop_l3': 0.11205522345570025, 'drop_l4': 0.17209441600169745, 'drop_l5': 0.6604219879542685, 'drop_l6': 0.6403416035672648, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:45:08,421] Trial 790 finished with value: 0.6187499761581421 and parameters: {'drop_l1': 0.10146804055027445, 'drop_l2': 0.16607361316138453, 'drop_l3': 0.10034522763431554, 'drop_l4': 0.17829183198682713, 'drop_l5': 0.5853178108455027, 'drop_l6': 0.676109977929695, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:45:11,645] Trial 791 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.10110994835595549, 'drop_l2': 0.1683445749207098, 'drop_l3': 0.11688484957430623, 'drop_l4': 0.26019780686693184, 'drop_l5': 0.5815709792210805, 'drop_l6': 0.6616117378251632, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:45:11,761] Trial 793 finished with value: 0.628125011920929 and parameters: {'drop_l1': 0.1273035003874141, 'drop_l2': 0.12997106072551393, 'drop_l3': 0.14639029780518473, 'drop_l4': 0.2570966402172334, 'drop_l5': 0.57751904969632, 'drop_l6': 0.6790864671782967, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:45:11,802] Trial 792 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.10035554405785188, 'drop_l2': 0.12916000908763742, 'drop_l3': 0.14322748391143147, 'drop_l4': 0.266428972590329, 'drop_l5': 0.581279508400699, 'drop_l6': 0.6754760659540124, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:45:48,019] Trial 794 finished with value: 0.609375 and parameters: {'drop_l1': 0.12535749469124305, 'drop_l2': 0.132711896158831, 'drop_l3': 0.14876076855378112, 'drop_l4': 0.2560313854660378, 'drop_l5': 0.6063643165399458, 'drop_l6': 0.6051997423394668, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:45:54,116] Trial 797 finished with value: 0.640625 and parameters: {'drop_l1': 0.12817140879551717, 'drop_l2': 0.11662567357066278, 'drop_l3': 0.22137787384921986, 'drop_l4': 0.21199022889538555, 'drop_l5': 0.6338903981584287, 'drop_l6': 0.6052436969161085, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:45:54,401] Trial 796 finished with value: 0.653124988079071 and parameters: {'drop_l1': 0.10007965386606316, 'drop_l2': 0.14817666575042426, 'drop_l3': 0.2298627383959223, 'drop_l4': 0.21309927247998342, 'drop_l5': 0.633321338759898, 'drop_l6': 0.6110148449247155, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:45:54,429] Trial 795 finished with value: 0.643750011920929 and parameters: {'drop_l1': 0.1283789822080097, 'drop_l2': 0.1298677235802401, 'drop_l3': 0.14636924507043295, 'drop_l4': 0.21144752731802133, 'drop_l5': 0.6067078223389649, 'drop_l6': 0.5800689816290271, 'activation': 'relu', 'kernel_init': 'lecun_uniform'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:46:14,117] Trial 798 finished with value: 0.6156250238418579 and parameters: {'drop_l1': 0.10055907572870199, 'drop_l2': 0.11248492473596063, 'drop_l3': 0.12630573935184283, 'drop_l4': 0.2141931915762567, 'drop_l5': 0.5428120156692826, 'drop_l6': 0.7625907243453074, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421. [I 2021-03-10 19:46:17,305] Trial 799 finished with value: 0.6312500238418579 and parameters: {'drop_l1': 0.42988709504346734, 'drop_l2': 0.15084514461133053, 'drop_l3': 0.12562334224099062, 'drop_l4': 0.2082334633848058, 'drop_l5': 0.6375066530098668, 'drop_l6': 0.5822794670226772, 'activation': 'relu', 'kernel_init': 'glorot_normal'}. Best is trial 154 with value: 0.6812499761581421.
study_coarse.best_value, study_coarse.best_params
(0.6812499761581421,
{'drop_l1': 0.11769241744475195,
'drop_l2': 0.1203918426967813,
'drop_l3': 0.3079277845699331,
'drop_l4': 0.16249139281479208,
'drop_l5': 0.3503972703488872,
'drop_l6': 0.5575415438784195,
'activation': 'relu',
'kernel_init': 'lecun_uniform'})
hp_score_board = hp_score_board.append({'Phase':'Coarse Tuning',
'Best Value':study_coarse.best_value,
'Hyperparameters':f'{study_coarse.best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.659375 | {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000} |
| 1 | Coarse Tuning | 0.681250 | {'drop_l1': 0.11769241744475195, 'drop_l2': 0.1203918426967813, 'drop_l3': 0.3079277845699331, 'drop_l4': 0.16249139281479208, 'drop_l5': 0.3503972703488872, 'drop_l6': 0.5575415438784195, 'activation': 'relu', 'kernel_init': 'lecun_uniform'} |
3% in accuracy with relu as the activation function and lecunn_uniform weight initialiser.study_coarse.trials_dataframe()
| number | value | datetime_start | datetime_complete | duration | params_activation | params_drop_l1 | params_drop_l2 | params_drop_l3 | params_drop_l4 | params_drop_l5 | params_drop_l6 | params_kernel_init | state | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.609375 | 2021-03-10 17:26:31.445669 | 2021-03-10 17:27:13.114369 | 0 days 00:00:41.668700 | tanh | 0.573417 | 0.293042 | 0.745083 | 0.444849 | 0.554822 | 0.230212 | lecun_uniform | COMPLETE |
| 1 | 1 | 0.456250 | 2021-03-10 17:26:31.447034 | 2021-03-10 17:27:11.789471 | 0 days 00:00:40.342437 | tanh | 0.711409 | 0.160361 | 0.400301 | 0.512967 | 0.280516 | 0.397131 | glorot_normal | COMPLETE |
| 2 | 2 | 0.562500 | 2021-03-10 17:26:31.451671 | 2021-03-10 17:27:12.774324 | 0 days 00:00:41.322653 | relu | 0.414262 | 0.802581 | 0.501053 | 0.442617 | 0.481107 | 0.279058 | lecun_uniform | COMPLETE |
| 3 | 3 | 0.421875 | 2021-03-10 17:26:31.457573 | 2021-03-10 17:27:12.719675 | 0 days 00:00:41.262102 | tanh | 0.375370 | 0.720826 | 0.262965 | 0.182611 | 0.635271 | 0.474187 | he_uniform | COMPLETE |
| 4 | 4 | 0.634375 | 2021-03-10 17:27:11.795386 | 2021-03-10 17:27:50.661062 | 0 days 00:00:38.865676 | relu | 0.436625 | 0.150704 | 0.184188 | 0.725575 | 0.291644 | 0.410675 | normal | COMPLETE |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 795 | 795 | 0.643750 | 2021-03-10 19:45:11.650297 | 2021-03-10 19:45:54.428540 | 0 days 00:00:42.778243 | relu | 0.128379 | 0.129868 | 0.146369 | 0.211448 | 0.606708 | 0.580069 | lecun_uniform | COMPLETE |
| 796 | 796 | 0.653125 | 2021-03-10 19:45:11.771837 | 2021-03-10 19:45:54.399918 | 0 days 00:00:42.628081 | relu | 0.100080 | 0.148177 | 0.229863 | 0.213099 | 0.633321 | 0.611015 | lecun_uniform | COMPLETE |
| 797 | 797 | 0.640625 | 2021-03-10 19:45:11.810830 | 2021-03-10 19:45:54.115933 | 0 days 00:00:42.305103 | relu | 0.128171 | 0.116626 | 0.221378 | 0.211990 | 0.633890 | 0.605244 | lecun_uniform | COMPLETE |
| 798 | 798 | 0.615625 | 2021-03-10 19:45:48.023221 | 2021-03-10 19:46:14.116471 | 0 days 00:00:26.093250 | relu | 0.100559 | 0.112485 | 0.126306 | 0.214193 | 0.542812 | 0.762591 | glorot_normal | COMPLETE |
| 799 | 799 | 0.631250 | 2021-03-10 19:45:54.122727 | 2021-03-10 19:46:17.304372 | 0 days 00:00:23.181645 | relu | 0.429887 | 0.150845 | 0.125623 | 0.208233 | 0.637507 | 0.582279 | glorot_normal | COMPLETE |
800 rows × 14 columns
optuna.visualization.plot_parallel_coordinate(study_coarse)
# optuna.visualization.plot_contour(study)
optuna.visualization.plot_optimization_history(study_coarse)
optuna.visualization.plot_slice(study_coarse)
coarse_best_params = study_coarse.best_params
coarse_best_params, study_coarse.best_value
({'drop_l1': 0.11769241744475195,
'drop_l2': 0.1203918426967813,
'drop_l3': 0.3079277845699331,
'drop_l4': 0.16249139281479208,
'drop_l5': 0.3503972703488872,
'drop_l6': 0.5575415438784195,
'activation': 'relu',
'kernel_init': 'lecun_uniform'},
0.6812499761581421)
drop_rate_per_layer =[coarse_best_params[f'drop_l{i}'] for i in range(1,arch_best_params['num_layers']+1)]
drop_rate_per_layer
coarse_best_params['drop_rate_per_layer'] = drop_rate_per_layer
for i in range(1,arch_best_params['num_layers']+1):
if f'drop_l{i}' in coarse_best_params.keys():
del coarse_best_params[f'drop_l{i}']
else:
pass
display(Markdown("#### Modified best parameters after Coarse Tuning :"))
print(coarse_best_params)
hp_best_params = {**arch_best_params , **coarse_best_params}
display(Markdown("#### Consolidate best parameters so far :"))
print(hp_best_params)
{'activation': 'relu', 'kernel_init': 'lecun_uniform', 'drop_rate_per_layer': [0.11769241744475195, 0.1203918426967813, 0.3079277845699331, 0.16249139281479208, 0.3503972703488872, 0.5575415438784195]}
{'num_layers': 6, 'arch': 'A D B', 'neurons_per_layer': [64, 256, 1000, 10, 1000, 1000], 'activation': 'relu', 'kernel_init': 'lecun_uniform', 'drop_rate_per_layer': [0.11769241744475195, 0.1203918426967813, 0.3079277845699331, 0.16249139281479208, 0.3503972703488872, 0.5575415438784195]}
def create_optimizer(trial):
# We optimize the choice of optimizers as well as their parameters.
kwargs = {}
optimizer_options = ["RMSprop", "Adam", "SGD", "Adadelta"]
optimizer_selected = trial.suggest_categorical("optimizer", optimizer_options)
if optimizer_selected == "RMSprop":
kwargs["learning_rate"] = trial.suggest_float("learning_rate", 1e-5, 1e-1, log=True)
kwargs["rho"] = trial.suggest_float("rho", 0.85, 0.99)
kwargs["momentum"] = trial.suggest_float("momentum", 1e-5, 1e-1, log=True)
kwargs['epsilon'] = trial.suggest_float("epsilon",1e-9, 1e-2, log=True)
elif optimizer_selected == "Adam":
kwargs["learning_rate"] = trial.suggest_float("learning_rate", 1e-5, 1e-1, log=True)
kwargs['epsilon'] = trial.suggest_float("epsilon",1e-9, 1e-2, log=True)
kwargs['beta_1'] = trial.suggest_float("beta_1", 0.85, 0.99)
kwargs['beta_2'] = trial.suggest_float("beta_2", 0.85, 0.99)
elif optimizer_selected == "SGD":
kwargs["learning_rate"] = trial.suggest_float("learning_rate", 1e-5, 1e-1, log=True)
kwargs["momentum"] = trial.suggest_float("momentum", 1e-5, 1e-1, log=True)
kwargs["nesterov"] = trial.suggest_categorical("nesterov",[True,False])
elif optimizer_selected == 'Adadelta':
kwargs["learning_rate"] = trial.suggest_float("learning_rate", 1e-5, 1e-1, log=True)
kwargs["rho"] = trial.suggest_float("rho", 0.85,0.99)
optimizer = getattr(tf.optimizers, optimizer_selected)(**kwargs)
return optimizer
def objective_fine(trial):
global hp_best_params,loss,metric,X_train,X_test,y_train,y_test
epochs = 50
optimizer = create_optimizer(trial)
kwargs = {'opts': optimizer,
'n_epochs' : epochs
}
kwargs = {**hp_best_params, **kwargs}
# print(kwargs)
dnn_1 = DNN_Model('classification')
m_1 = dnn_1.create_model(X_train.shape[1],n_class,**kwargs)
# callbacks = LivePlot(2,train_loss='mean_squared_error',train_metric='mean_squared_error')
callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,['sparse_categorical_crossentropy'],['accuracy'],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
return result[1]
study_fine = optuna.create_study(direction='maximize',sampler=optuna.samplers.TPESampler())
study_fine.optimize(objective_fine,n_trials=600,n_jobs=-13)
[I 2021-03-10 19:46:18,771] A new study created in memory with name: no-name-eaf0701d-90e8-4f2d-8953-191d1354da56 [I 2021-03-10 19:46:56,865] Trial 1 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0005744618544311334, 'momentum': 0.08831832852533344, 'nesterov': True}. Best is trial 1 with value: 0.6156250238418579. [I 2021-03-10 19:46:59,672] Trial 0 finished with value: 0.5874999761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.07820952154288607, 'momentum': 2.6236948645999348e-05, 'nesterov': True}. Best is trial 1 with value: 0.6156250238418579. [I 2021-03-10 19:47:00,042] Trial 3 finished with value: 0.578125 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 2.0635667269637963e-05, 'rho': 0.8911619991198516, 'momentum': 0.02315578336246564, 'epsilon': 4.189703052960511e-09}. Best is trial 1 with value: 0.6156250238418579. [I 2021-03-10 19:47:00,526] Trial 2 finished with value: 0.453125 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.034561638553711166, 'rho': 0.9178193753197892, 'momentum': 9.608343584931195e-05, 'epsilon': 4.427008845259751e-05}. Best is trial 1 with value: 0.6156250238418579. [I 2021-03-10 19:47:34,110] Trial 4 finished with value: 0.6000000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.007482279914893651, 'epsilon': 0.0002474035745327583, 'beta_1': 0.8516303345458639, 'beta_2': 0.872424413503256}. Best is trial 1 with value: 0.6156250238418579. [I 2021-03-10 19:47:37,304] Trial 5 finished with value: 0.574999988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.060140700391158854, 'momentum': 0.0018987175783091229, 'nesterov': False}. Best is trial 1 with value: 0.6156250238418579. [I 2021-03-10 19:47:38,551] Trial 6 finished with value: 0.612500011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.015924055113254412, 'rho': 0.8602132633669782, 'momentum': 5.7057160187103964e-05, 'epsilon': 0.0001025791879066707}. Best is trial 1 with value: 0.6156250238418579. [I 2021-03-10 19:47:38,579] Trial 7 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0001143486690097398, 'epsilon': 6.264264227244593e-07, 'beta_1': 0.9012465001300498, 'beta_2': 0.9528596299364086}. Best is trial 7 with value: 0.6187499761581421. [I 2021-03-10 19:48:11,204] Trial 8 finished with value: 0.637499988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004157240037746191, 'rho': 0.9503633407118786, 'momentum': 1.0058996518142282e-05, 'epsilon': 0.0009138991057147599}. Best is trial 8 with value: 0.637499988079071. [I 2021-03-10 19:48:16,371] Trial 9 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.006179633196053471, 'rho': 0.9383864948213767}. Best is trial 8 with value: 0.637499988079071. [I 2021-03-10 19:48:17,058] Trial 10 finished with value: 0.512499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.032226773823011125, 'epsilon': 4.564399521700133e-05, 'beta_1': 0.8685764333370595, 'beta_2': 0.9163735476472191}. Best is trial 8 with value: 0.637499988079071. [I 2021-03-10 19:48:17,214] Trial 11 finished with value: 0.671875 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0010481734929331486, 'momentum': 0.009287368339457537, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:48:47,146] Trial 12 finished with value: 0.04374999925494194 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0006766782623807245, 'rho': 0.8743844497511072}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:48:56,358] Trial 13 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005869206135780043, 'rho': 0.9876988650002966, 'momentum': 0.001394140444643469, 'epsilon': 0.002557941738563556}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:48:57,360] Trial 15 finished with value: 0.4375 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0008420329231615127, 'rho': 0.9871185959128048}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:48:57,919] Trial 14 finished with value: 0.4593749940395355 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0020234343206689247, 'rho': 0.9829521744383696}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:49:20,224] Trial 16 finished with value: 0.546875 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00011317275220159536, 'momentum': 0.004940795707577011, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:49:35,055] Trial 17 finished with value: 0.5687500238418579 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00013864826313381983, 'momentum': 0.010040143747122112, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:49:36,972] Trial 18 finished with value: 0.512499988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00011172570449075804, 'momentum': 0.012469117830198157, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:49:36,973] Trial 19 finished with value: 0.45625001192092896 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00010957096262118084, 'momentum': 0.008459150055950593, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:49:53,014] Trial 20 finished with value: 0.6031249761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00015813235766678902, 'rho': 0.9538775699253503, 'momentum': 0.013680971997097394, 'epsilon': 0.004597347412886781}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:50:16,536] Trial 21 finished with value: 0.59375 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 2.1802293711726038e-05, 'rho': 0.9495114403395211, 'momentum': 0.00023778088073407805, 'epsilon': 0.006720512543890851}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:50:19,524] Trial 23 finished with value: 0.640625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.002474354359845261, 'rho': 0.9493804536087775, 'momentum': 0.0002272152308604665, 'epsilon': 0.009299466855164668}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:50:19,627] Trial 22 finished with value: 0.621874988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.002416023502710064, 'rho': 0.9526617949697378, 'momentum': 0.0002840045993175347, 'epsilon': 0.006544806359587672}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:50:27,838] Trial 24 finished with value: 0.59375 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00242081332227552, 'rho': 0.9623259208628001, 'momentum': 0.00048375469573983746, 'epsilon': 0.008789528371749893}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:50:58,148] Trial 25 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0021346067281117, 'rho': 0.971935096402985, 'momentum': 0.0005460337172995676, 'epsilon': 0.008567864082738024}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:51:02,352] Trial 26 finished with value: 0.625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.002356129548739792, 'rho': 0.9644440079908729, 'momentum': 1.0351085658251662e-05, 'epsilon': 8.59304987790824e-08}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:51:02,420] Trial 27 finished with value: 0.596875011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.002484401728086351, 'rho': 0.9219411269610999, 'momentum': 0.07534300409237872, 'epsilon': 7.614833354259876e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:51:05,140] Trial 28 finished with value: 0.668749988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00032088782388483357, 'rho': 0.9185326962529091, 'momentum': 1.1124582110784646e-05, 'epsilon': 6.495602603989958e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:51:35,944] Trial 29 finished with value: 0.659375011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003562877691230864, 'rho': 0.9234865584748976, 'momentum': 1.8045372623389664e-05, 'epsilon': 3.857643088444074e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:51:43,610] Trial 31 finished with value: 0.637499988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0002888386000527471, 'momentum': 0.002955210659229714, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:51:44,278] Trial 30 finished with value: 0.621874988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00026974497453845133, 'rho': 0.9273240713304212, 'momentum': 0.08432034116787757, 'epsilon': 0.0008772182118955055}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:51:45,591] Trial 32 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00029174696916451276, 'momentum': 0.003165493648998007, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:52:06,242] Trial 33 finished with value: 0.6031249761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0002887801648999077, 'momentum': 2.5824198078732282e-05, 'nesterov': True}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:52:25,491] Trial 36 finished with value: 0.621874988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 4.791574839778362e-05, 'rho': 0.9021985501588671, 'momentum': 2.574555609463644e-05, 'epsilon': 5.2650742034908364e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:52:25,547] Trial 34 finished with value: 0.637499988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0011824570911134176, 'rho': 0.9151127529687115, 'momentum': 3.0008752326613877e-05, 'epsilon': 2.0531692279654283e-08}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:52:25,577] Trial 35 finished with value: 0.578125 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.001205350756924779, 'rho': 0.9048679784514244, 'momentum': 2.273505060745777e-05, 'epsilon': 2.2343589976550442e-08}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:52:37,081] Trial 37 finished with value: 0.621874988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 4.502106353877949e-05, 'rho': 0.9023648232409512, 'momentum': 2.3673445141619114e-05, 'epsilon': 5.449611401170057e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:53:08,145] Trial 38 finished with value: 0.59375 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0013098125698920813, 'rho': 0.9039181832563081, 'momentum': 7.123651518305912e-05, 'epsilon': 2.6906605747751965e-08}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:53:08,147] Trial 39 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.001309976946669697, 'epsilon': 5.2886168656648685e-06, 'beta_1': 0.9828330494753743, 'beta_2': 0.861700782780313}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:53:08,161] Trial 40 finished with value: 0.5531250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.005487196686537305, 'epsilon': 7.315289579814858e-06, 'beta_1': 0.9836916450920052, 'beta_2': 0.983365525367178}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:53:12,263] Trial 41 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0047761480878177375, 'epsilon': 8.214464385680592e-07, 'beta_1': 0.9893224120306927, 'beta_2': 0.9800605685858031}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:53:49,527] Trial 43 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00045039243871752626, 'epsilon': 4.708341137583535e-07, 'beta_1': 0.9875424813211526, 'beta_2': 0.8551638715952676}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:53:50,585] Trial 45 finished with value: 0.48124998807907104 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.012224916859480845, 'epsilon': 9.140462774300005e-07, 'beta_1': 0.9864172051827373, 'beta_2': 0.8543066698043162}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:53:51,346] Trial 44 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006987592071704853, 'epsilon': 6.092791756860043e-07, 'beta_1': 0.9756550386281974, 'beta_2': 0.8597185628367815}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:53:51,631] Trial 42 finished with value: 0.5 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0051379334508443045, 'epsilon': 7.321932619879611e-07, 'beta_1': 0.9859257343572242, 'beta_2': 0.8532981314900969}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:54:30,456] Trial 46 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00042699034864994683, 'epsilon': 2.3043427116327946e-07, 'beta_1': 0.9554380151420994, 'beta_2': 0.8718362453985208}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:54:32,624] Trial 49 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004585387049794076, 'epsilon': 1.6301444634750797e-05, 'beta_1': 0.9446118714993513, 'beta_2': 0.8973651597988996}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:54:32,647] Trial 48 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005272015217711685, 'epsilon': 1.5916387817554304e-07, 'beta_1': 0.9469837030446302, 'beta_2': 0.8964327165746946}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:54:32,675] Trial 47 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004821264435564925, 'epsilon': 1.055966691782276e-07, 'beta_1': 0.9487429678148134, 'beta_2': 0.8845038792420759}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:55:09,203] Trial 50 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004393834439197572, 'epsilon': 1.7098486355844657e-05, 'beta_1': 0.947212872534559, 'beta_2': 0.8934643220744142}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:55:13,115] Trial 51 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0001909300471124391, 'epsilon': 2.1785197714976266e-06, 'beta_1': 0.9263137103388847, 'beta_2': 0.8914379095687053}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:55:13,154] Trial 52 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00021432677584482435, 'epsilon': 2.127789865882382e-06, 'beta_1': 0.9184746788538419, 'beta_2': 0.9085267626500676}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:55:13,160] Trial 53 finished with value: 0.07500000298023224 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0001849498980410735, 'rho': 0.8817907566477335}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:55:41,409] Trial 54 finished with value: 0.606249988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0001722227604238101, 'epsilon': 1.751073453665754e-07, 'beta_1': 0.9594203135253405, 'beta_2': 0.9098078066515336}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:55:53,133] Trial 56 finished with value: 0.596875011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008913920612972289, 'epsilon': 1.1959295216927134e-07, 'beta_1': 0.964019746314339, 'beta_2': 0.9301194987601235}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:55:54,041] Trial 55 finished with value: 0.612500011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009488429068662391, 'epsilon': 1.742095047912833e-07, 'beta_1': 0.9625903926402855, 'beta_2': 0.9151278390602791}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:55:54,052] Trial 57 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008769584274417332, 'epsilon': 2.0964132137205995e-07, 'beta_1': 0.9654109256857595, 'beta_2': 0.9395116678877818}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:56:10,258] Trial 58 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000975245732014464, 'epsilon': 3.5948265081132e-07, 'beta_1': 0.9702322499092452, 'beta_2': 0.9354064995607929}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:56:33,194] Trial 60 finished with value: 0.621874988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0015378819560493513, 'momentum': 0.02567157406221694, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:56:33,205] Trial 59 finished with value: 0.4749999940395355 and parameters: {'optimizer': 'SGD', 'learning_rate': 8.270996155003393e-05, 'momentum': 0.04405579601504631, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:56:33,217] Trial 61 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0034798159071873142, 'momentum': 0.00012240369268803707, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:56:42,098] Trial 62 finished with value: 0.637499988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0013401456963551687, 'momentum': 0.030402929218956726, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:57:14,562] Trial 64 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006232951429888513, 'epsilon': 5.046670505724936e-08, 'beta_1': 0.9732546217464191, 'beta_2': 0.9575588012109875}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:57:16,046] Trial 63 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006221715229519341, 'epsilon': 1.9630940482188732e-06, 'beta_1': 0.9740464630084997, 'beta_2': 0.952927517714315}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:57:16,048] Trial 65 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006798896917812881, 'epsilon': 1.9392721988777574e-09, 'beta_1': 0.9753159939397342, 'beta_2': 0.9590988666540763}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:57:20,209] Trial 66 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005559027901282701, 'epsilon': 4.280163014773002e-08, 'beta_1': 0.9302686827932088, 'beta_2': 0.8734905542545605}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:57:59,998] Trial 70 finished with value: 0.10000000149011612 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.00035388915124297905, 'rho': 0.930349528262186}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:58:00,216] Trial 67 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0016587028633178122, 'epsilon': 1.9833055134375142e-06, 'beta_1': 0.9327683700481704, 'beta_2': 0.8704688694466662}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:58:00,588] Trial 68 finished with value: 0.36250001192092896 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0016403698238169092, 'rho': 0.9279137743797967}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:58:00,667] Trial 69 finished with value: 0.4375 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0017004565903470123, 'rho': 0.9359954855124124}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:58:41,412] Trial 71 finished with value: 0.565625011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0016609160237628523, 'epsilon': 3.657940315533792e-07, 'beta_1': 0.9380478207130524, 'beta_2': 0.927583562979096}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:58:43,926] Trial 74 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035938582192479563, 'epsilon': 3.9407682817824387e-07, 'beta_1': 0.9539300632023853, 'beta_2': 0.8627256602543055}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:58:44,203] Trial 72 finished with value: 0.643750011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0032636423685119603, 'rho': 0.9386891527854798, 'momentum': 1.356294382283223e-05, 'epsilon': 2.4697256585251995e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:58:44,257] Trial 73 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00033459164467615406, 'rho': 0.9142917382490598, 'momentum': 1.2237900169249169e-05, 'epsilon': 3.703631315048324e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:59:22,459] Trial 75 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00038424395103639914, 'epsilon': 3.6012869791725396e-07, 'beta_1': 0.9552821710503477, 'beta_2': 0.8607567618267177}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:59:25,687] Trial 78 finished with value: 0.596875011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010506568163958527, 'epsilon': 8.708982441969476e-09, 'beta_1': 0.9689342655892556, 'beta_2': 0.8792691584417458}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:59:25,706] Trial 77 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003404192189809494, 'epsilon': 4.1648793756998803e-07, 'beta_1': 0.9898338990295414, 'beta_2': 0.8627093301909371}. Best is trial 11 with value: 0.671875. [I 2021-03-10 19:59:25,731] Trial 76 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003574319671592364, 'epsilon': 3.5345059808240967e-07, 'beta_1': 0.9898130648142982, 'beta_2': 0.86337517965966}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:00:00,809] Trial 79 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00024402990390055173, 'epsilon': 1.4065115855869472e-05, 'beta_1': 0.9534784621745741, 'beta_2': 0.882200883229807}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:00:05,600] Trial 80 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00024350097692146263, 'epsilon': 1.7528981871824808e-05, 'beta_1': 0.9095847954382277, 'beta_2': 0.8741438261917632}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:00:06,784] Trial 82 finished with value: 0.653124988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002331349219702583, 'rho': 0.8534164914487293, 'momentum': 0.0009654297340164802, 'epsilon': 4.147535663626125e-05}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:00:07,064] Trial 81 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00024253658261681683, 'rho': 0.8912045329826734, 'momentum': 0.0008443667618613502, 'epsilon': 1.0106162439801445e-05}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:00:34,129] Trial 83 finished with value: 0.609375 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0007712874131653726, 'rho': 0.8514969242901982, 'momentum': 0.0009243523039821827, 'epsilon': 6.855541981333989e-05}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:00:46,795] Trial 84 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00014140109937020928, 'epsilon': 5.319144402946503e-05, 'beta_1': 0.979705642501643, 'beta_2': 0.8516593013319635}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:00:49,170] Trial 85 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00048607169422204685, 'epsilon': 1.2157432855778949e-06, 'beta_1': 0.9780992952281102, 'beta_2': 0.8504333446104703}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:00:49,179] Trial 86 finished with value: 0.596875011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 9.136370409029755e-05, 'rho': 0.8536873728692231, 'momentum': 0.0010596266232646948, 'epsilon': 8.260034972258497e-05}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:01:07,129] Trial 87 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00012122931194428083, 'rho': 0.8894635963427309, 'momentum': 4.903133365993471e-05, 'epsilon': 1.3316366683564235e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:01:28,631] Trial 88 finished with value: 0.640625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 7.645261722477878e-05, 'rho': 0.867721422925507, 'momentum': 0.002035477154181311, 'epsilon': 2.7091621977677738e-05}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:01:31,552] Trial 90 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005360369134706424, 'rho': 0.8909061695964834, 'momentum': 4.215315253905836e-05, 'epsilon': 3.875926433363817e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:01:31,556] Trial 89 finished with value: 0.643750011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 9.881401709643206e-05, 'rho': 0.8886388655307531, 'momentum': 0.006076047600181872, 'epsilon': 3.442641375411401e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:01:39,918] Trial 91 finished with value: 0.5874999761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.001098539694166316, 'rho': 0.8683426484288159, 'momentum': 0.0001253537988716015, 'epsilon': 3.3550069173595444e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:02:07,886] Trial 92 finished with value: 0.612500011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0010767072513090162, 'rho': 0.8922956035305706, 'momentum': 0.00012409713641199348, 'epsilon': 4.006547935234888e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:02:13,255] Trial 93 finished with value: 0.65625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00021811721266208853, 'rho': 0.8744683336084721, 'momentum': 0.00014447269206651313, 'epsilon': 0.0003037713740296391}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:02:13,384] Trial 94 finished with value: 0.640625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00020993016393146805, 'rho': 0.8662787787320668, 'momentum': 9.708249988221412e-05, 'epsilon': 1.036293310505322e-05}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:02:17,613] Trial 95 finished with value: 0.640625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00022261134423915156, 'rho': 0.9099048224211534, 'momentum': 0.0006363699590169066, 'epsilon': 0.00032829569599763505}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:02:43,003] Trial 96 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002448037368520006, 'rho': 0.9124496504124436, 'momentum': 0.0008822423187579417, 'epsilon': 0.00021000602692917771}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:02:54,565] Trial 97 finished with value: 0.621874988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002175703038687093, 'rho': 0.8813780108881083, 'momentum': 0.0006612197178120408, 'epsilon': 0.00020310986375249196}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:02:55,715] Trial 98 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00014926525891868398, 'rho': 0.8770568103929769, 'momentum': 0.0004323288206812566, 'epsilon': 0.00012967698397684234}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:02:56,671] Trial 99 finished with value: 0.59375 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0007454172845433671, 'momentum': 0.00019999916672562592, 'nesterov': True}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:03:14,105] Trial 100 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00015515560674596218, 'rho': 0.8773768821389315, 'momentum': 0.0003062653583127065, 'epsilon': 0.0018012216824074953}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:03:36,167] Trial 102 finished with value: 0.606249988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00038066027914647005, 'momentum': 1.4951435634816177e-05, 'nesterov': True}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:03:37,225] Trial 101 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0007799718487908733, 'momentum': 0.0002967349666003787, 'nesterov': True}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:03:37,294] Trial 103 finished with value: 0.625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00028501608243022163, 'rho': 0.8595018739298667, 'momentum': 0.0003080484749861615, 'epsilon': 0.0006917470005148447}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:03:45,308] Trial 104 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004001517030067839, 'epsilon': 1.1329899960639348e-07, 'beta_1': 0.9555700837138863, 'beta_2': 0.8669482498587431}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:04:19,856] Trial 107 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00046678610807360975, 'epsilon': 6.269143552711781e-07, 'beta_1': 0.9552819816614858, 'beta_2': 0.9422911226741194}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:04:19,928] Trial 106 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002894010270071585, 'rho': 0.858561012659699, 'momentum': 0.00136758119543784, 'epsilon': 7.440747219858483e-08}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:04:19,939] Trial 105 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00029412913187221764, 'epsilon': 7.640798416563552e-08, 'beta_1': 0.9561425646473384, 'beta_2': 0.8682510736988873}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:04:23,140] Trial 108 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005461068419536767, 'rho': 0.8604622478974256, 'momentum': 0.0018638569479886833, 'epsilon': 7.44241100688334e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:05:00,351] Trial 109 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005377055295790228, 'epsilon': 5.935556636978699e-07, 'beta_1': 0.9433531663493432, 'beta_2': 0.9413069804907283}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:05:02,164] Trial 111 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005585772449775872, 'epsilon': 1.2570357156359028e-06, 'beta_1': 0.9399472263975681, 'beta_2': 0.8545197586365207}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:05:02,637] Trial 110 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005763783173132144, 'epsilon': 7.712078789394763e-06, 'beta_1': 0.8880772347707383, 'beta_2': 0.8570758107996955}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:05:03,291] Trial 112 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0009316165813304864, 'rho': 0.9201960221335151, 'momentum': 0.0030429117389851055, 'epsilon': 2.9456154530438047e-05}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:05:43,145] Trial 113 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009081064031802743, 'epsilon': 1.1768300150481545e-06, 'beta_1': 0.9386003064587582, 'beta_2': 0.9421583431723048}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:05:44,130] Trial 116 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004163291583608352, 'epsilon': 1.9309448724483875e-07, 'beta_1': 0.9839146569624418, 'beta_2': 0.9007467858958432}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:05:45,076] Trial 115 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004321243045396357, 'epsilon': 5.833187244948883e-07, 'beta_1': 0.9701364735499907, 'beta_2': 0.9411292440283805}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:05:45,225] Trial 114 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000454141299243357, 'epsilon': 8.198474287554638e-07, 'beta_1': 0.8695027042395036, 'beta_2': 0.9398919362387116}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:06:24,865] Trial 118 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00032457497509991726, 'epsilon': 2.708123281636424e-07, 'beta_1': 0.9686632207084294, 'beta_2': 0.8870758787813864}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:06:25,778] Trial 119 finished with value: 0.581250011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0014228410797243933, 'epsilon': 2.16506155550855e-07, 'beta_1': 0.9822688183571503, 'beta_2': 0.8972092281666026}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:06:26,708] Trial 120 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00132491036581718, 'epsilon': 2.9660579734768496e-07, 'beta_1': 0.8510965188408588, 'beta_2': 0.9671958507039125}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:06:26,843] Trial 117 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00041004768757455645, 'epsilon': 1.606337649875111e-07, 'beta_1': 0.9827375198282392, 'beta_2': 0.8589780083201034}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:07:05,040] Trial 121 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00198254569277474, 'epsilon': 2.635020242965621e-07, 'beta_1': 0.8623572938576135, 'beta_2': 0.9307461539451648}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:07:07,027] Trial 122 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00018485862075115462, 'epsilon': 1.4994781644178664e-07, 'beta_1': 0.8578370703034482, 'beta_2': 0.9282141519955085}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:07:07,393] Trial 124 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006986037804204188, 'epsilon': 7.901928315695978e-07, 'beta_1': 0.9499459561580768, 'beta_2': 0.9311247734322019}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:07:07,640] Trial 123 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000171828833070772, 'rho': 0.896687073543397, 'momentum': 1.871248196794614e-05, 'epsilon': 2.440577033732446e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:07:44,420] Trial 125 finished with value: 0.612500011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035137860382018315, 'epsilon': 9.165692363219448e-07, 'beta_1': 0.8852321194977266, 'beta_2': 0.8882281151169799}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:07:47,311] Trial 126 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0001722527123100596, 'rho': 0.8990059388032501, 'momentum': 0.015701369811492037, 'epsilon': 8.784115484595912e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:07:47,418] Trial 128 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00031688458873557616, 'epsilon': 9.029036612568153e-07, 'beta_1': 0.8740108114715017, 'beta_2': 0.922336874060842}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:07:47,436] Trial 127 finished with value: 0.612500011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00018241469865756826, 'epsilon': 8.858553192318059e-07, 'beta_1': 0.8670122089488121, 'beta_2': 0.9237728659796088}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:08:23,672] Trial 129 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 6.516198828245172e-05, 'epsilon': 1.6857390900006636e-06, 'beta_1': 0.8591541583591918, 'beta_2': 0.9224679297768792}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:08:28,156] Trial 132 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007187464552707333, 'epsilon': 1.6242713281358544e-06, 'beta_1': 0.8595222469543917, 'beta_2': 0.9173518113844731}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:08:28,227] Trial 130 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 6.599005186164587e-05, 'epsilon': 1.4603451945828752e-06, 'beta_1': 0.8620780439381928, 'beta_2': 0.9208246222214406}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:08:28,237] Trial 131 finished with value: 0.612500011920929 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.000671222306340874, 'momentum': 0.018576743347057695, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:08:56,281] Trial 133 finished with value: 0.46875 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.07390279683118345, 'epsilon': 4.877987707171546e-08, 'beta_1': 0.961801325910017, 'beta_2': 0.9350822427550204}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:09:08,045] Trial 135 finished with value: 0.5249999761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00011266206940283251, 'momentum': 0.004959503242207379, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:09:08,078] Trial 134 finished with value: 0.5 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00012344840951097509, 'momentum': 0.014294564477751924, 'nesterov': False}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:09:08,085] Trial 136 finished with value: 0.606249988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0001211799627809049, 'rho': 0.8996561538543143, 'momentum': 0.007618683251179619, 'epsilon': 1.3647372690065926e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:09:26,113] Trial 137 finished with value: 0.421875 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.00013102999654254825, 'rho': 0.8962646838196977}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:09:49,891] Trial 139 finished with value: 0.671875 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002729188746142657, 'rho': 0.909831521313532, 'momentum': 0.04147493015968138, 'epsilon': 4.6431647037072496e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:09:51,179] Trial 140 finished with value: 0.590624988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00025512255093452244, 'rho': 0.9087069698155497, 'momentum': 0.03541568173867646, 'epsilon': 4.77201544978838e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:09:51,211] Trial 138 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0001966900056717443, 'rho': 0.8966918464302106, 'momentum': 0.007880634822126706, 'epsilon': 4.570537868968024e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:09:58,791] Trial 141 finished with value: 0.643750011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00027147483805061817, 'rho': 0.9080257146772587, 'momentum': 4.0607351755837754e-05, 'epsilon': 5.130243848686677e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:10:30,071] Trial 142 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002564806176574753, 'epsilon': 4.537229982086266e-07, 'beta_1': 0.948268152366208, 'beta_2': 0.9109709610337167}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:10:32,247] Trial 143 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003062180301214758, 'rho': 0.9224325849251024, 'momentum': 0.09999533313586509, 'epsilon': 2.66602217126243e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:10:32,721] Trial 144 finished with value: 0.65625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002925973768126991, 'rho': 0.9218222726103122, 'momentum': 0.04359214288983165, 'epsilon': 2.8390395556775125e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:10:36,003] Trial 145 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00030861498062387564, 'rho': 0.923645473551353, 'momentum': 0.04440305659793467, 'epsilon': 3.254252501570968e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:11:12,282] Trial 146 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003284962967052715, 'rho': 0.9238508908293858, 'momentum': 0.011311714557240296, 'epsilon': 2.6556958205888947e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:11:14,916] Trial 147 finished with value: 0.653124988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00017722653645617568, 'rho': 0.9432711834112313, 'momentum': 0.06387291651742455, 'epsilon': 2.876075785096563e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:11:15,659] Trial 148 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00018939502186763802, 'rho': 0.8838506741359692, 'momentum': 0.0654458950278945, 'epsilon': 9.767068056651438e-08}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:11:15,778] Trial 149 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00017232178031593819, 'epsilon': 8.442529562294695e-08, 'beta_1': 0.9169027704317607, 'beta_2': 0.9357835997402226}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:11:55,661] Trial 150 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0008147391160147126, 'rho': 0.9419873773308746, 'momentum': 0.06285029604780018, 'epsilon': 6.164029579118501e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:11:59,630] Trial 153 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008368842237712388, 'epsilon': 3.534331452677895e-08, 'beta_1': 0.9195062972472846, 'beta_2': 0.9495463999750934}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:12:00,220] Trial 152 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004706046900746581, 'rho': 0.8832615500118082, 'momentum': 0.022330390371252718, 'epsilon': 8.693305129377568e-08}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:12:00,243] Trial 151 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004712681279800651, 'rho': 0.9170245791438699, 'momentum': 0.0685855217327948, 'epsilon': 7.486695965798525e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:12:36,551] Trial 154 finished with value: 0.625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004636233116030658, 'rho': 0.9173843450596857, 'momentum': 0.01687900946387251, 'epsilon': 1.396329967760809e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:12:43,770] Trial 157 finished with value: 0.653124988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00020824368832572882, 'rho': 0.885395029902131, 'momentum': 0.0040400125943093395, 'epsilon': 1.0515233465700655e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:12:44,248] Trial 156 finished with value: 0.640625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00020893489829344262, 'rho': 0.9183614253796073, 'momentum': 0.016373088144936803, 'epsilon': 1.2172162928468493e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:12:44,368] Trial 155 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00015273761635662344, 'rho': 0.8825801919854899, 'momentum': 0.016873807152257562, 'epsilon': 6.79442125651035e-08}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:13:08,583] Trial 158 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000204517173623922, 'epsilon': 1.0829444649023913e-07, 'beta_1': 0.9674053496766467, 'beta_2': 0.9031544109333128}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:13:26,012] Trial 160 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0011162682232185217, 'epsilon': 1.0551603375878245e-05, 'beta_1': 0.9670571775949587, 'beta_2': 0.9467192807777888}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:13:27,145] Trial 161 finished with value: 0.590624988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005959467315796253, 'rho': 0.934820027053043, 'momentum': 0.04850799854810799, 'epsilon': 5.382012088104765e-06}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:13:27,184] Trial 159 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0001527266185691188, 'epsilon': 6.200153107292438e-08, 'beta_1': 0.9674448086751128, 'beta_2': 0.9346132344332201}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:13:41,506] Trial 162 finished with value: 0.609375 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003876348518657612, 'epsilon': 1.695157196169109e-07, 'beta_1': 0.9720576165741941, 'beta_2': 0.9283344741806936}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:14:07,084] Trial 163 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006362771217667487, 'epsilon': 2.0669238643485505e-07, 'beta_1': 0.9596815001114233, 'beta_2': 0.9344411161349435}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:14:09,172] Trial 165 finished with value: 0.625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003836522140064059, 'rho': 0.9061650109083185, 'momentum': 0.033073856715707665, 'epsilon': 1.9353861758979358e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:14:09,245] Trial 164 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00035515349809939445, 'rho': 0.9315404660724454, 'momentum': 0.02641594320568019, 'epsilon': 1.5909110854377247e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:14:15,254] Trial 166 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002502528265799125, 'epsilon': 2.3237070818471495e-07, 'beta_1': 0.9048352574140054, 'beta_2': 0.9349522249416548}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:14:46,489] Trial 167 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006669024611209033, 'epsilon': 2.014743247101043e-07, 'beta_1': 0.9600515383051069, 'beta_2': 0.9349134760356286}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:14:50,779] Trial 168 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010910515129855143, 'epsilon': 3.533687390071912e-08, 'beta_1': 0.9598447563096756, 'beta_2': 0.935763741517536}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:14:51,863] Trial 169 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 1.2360258884421323e-05, 'rho': 0.9115360822114016, 'momentum': 0.09613996484425223, 'epsilon': 5.071448141392591e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:14:53,054] Trial 170 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006414460213589845, 'rho': 0.9014642930206079, 'momentum': 1.0267509578261491e-05, 'epsilon': 3.9706361599369413e-07}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:15:18,800] Trial 171 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006293483658775804, 'epsilon': 3.982491543935686e-07, 'beta_1': 0.9597139110559288, 'beta_2': 0.9378869540686767}. Best is trial 11 with value: 0.671875. [I 2021-03-10 20:15:30,235] Trial 172 finished with value: 0.675000011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00059529702050015, 'epsilon': 4.1456390888971527e-07, 'beta_1': 0.9503882330886023, 'beta_2': 0.9470604204811185}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:15:31,902] Trial 173 finished with value: 0.671875 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006627621553537945, 'epsilon': 4.216551195937759e-07, 'beta_1': 0.9488514723958879, 'beta_2': 0.9467545033700495}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:15:31,924] Trial 174 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009496901413031522, 'epsilon': 3.405442420839025e-07, 'beta_1': 0.960387197181003, 'beta_2': 0.9280830919705746}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:15:47,559] Trial 175 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007416031585289559, 'epsilon': 7.210654841878101e-07, 'beta_1': 0.9515179145854105, 'beta_2': 0.9470572626056414}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:16:09,862] Trial 176 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007505442502322385, 'epsilon': 8.431017008553585e-07, 'beta_1': 0.9499470561907425, 'beta_2': 0.953092523054439}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:16:11,133] Trial 178 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007465354133248619, 'epsilon': 6.608396006342433e-07, 'beta_1': 0.950099640342595, 'beta_2': 0.9476184159031134}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:16:11,150] Trial 177 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007068915861342348, 'epsilon': 6.903592241372724e-07, 'beta_1': 0.9497003533789334, 'beta_2': 0.9477003387518915}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:16:19,599] Trial 179 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005428070279234742, 'epsilon': 6.713061284998865e-07, 'beta_1': 0.945447073637438, 'beta_2': 0.9553542300694711}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:16:50,452] Trial 180 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005072249962290047, 'epsilon': 5.268162090557197e-07, 'beta_1': 0.9439730936924069, 'beta_2': 0.9595336143743751}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:16:50,975] Trial 182 finished with value: 0.609375 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005134096757202966, 'epsilon': 2.3529927153237757e-07, 'beta_1': 0.9452423741102441, 'beta_2': 0.9623335223602936}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:16:50,976] Trial 181 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005346063391562988, 'epsilon': 2.9135676322029832e-06, 'beta_1': 0.9630630630487225, 'beta_2': 0.8787308576386257}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:16:53,935] Trial 183 finished with value: 0.668749988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005490222095160583, 'epsilon': 2.3099404816125347e-07, 'beta_1': 0.9441334167508497, 'beta_2': 0.9647001153131441}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:17:30,178] Trial 184 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0012780991310465696, 'epsilon': 2.3582305631208267e-07, 'beta_1': 0.9344743147374146, 'beta_2': 0.9611985460085366}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:17:32,663] Trial 187 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0013652417760538773, 'epsilon': 3.5679895339642363e-07, 'beta_1': 0.931631419000226, 'beta_2': 0.9438145042000241}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:17:32,675] Trial 185 finished with value: 0.612500011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0013243520956993922, 'epsilon': 1.182515822346544e-06, 'beta_1': 0.93369533072563, 'beta_2': 0.9538280723117964}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:17:33,265] Trial 186 finished with value: 0.609375 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0013133805485437808, 'epsilon': 1.2935078706882187e-06, 'beta_1': 0.9573412932950802, 'beta_2': 0.9543092939883644}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:18:10,173] Trial 188 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009245784896962472, 'epsilon': 3.599944092719044e-07, 'beta_1': 0.9414238272125843, 'beta_2': 0.9690307872942703}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:18:15,150] Trial 189 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009630938337048032, 'epsilon': 1.7420675035301695e-06, 'beta_1': 0.9414715350687349, 'beta_2': 0.9722357081739654}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:18:15,627] Trial 190 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004335850101499042, 'epsilon': 8.556500393758802e-06, 'beta_1': 0.9409203138559937, 'beta_2': 0.951000603236445}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:18:15,713] Trial 191 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004144268991038883, 'epsilon': 1.7356973605416838e-06, 'beta_1': 0.9411102816330819, 'beta_2': 0.9563313088241778}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:18:47,114] Trial 192 finished with value: 0.23749999701976776 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0004139193871748872, 'rho': 0.8937821974900321}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:18:57,041] Trial 194 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006014763415113932, 'epsilon': 5.010453067542342e-07, 'beta_1': 0.9780365067331406, 'beta_2': 0.9327023413228851}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:18:58,090] Trial 195 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006220543786263316, 'epsilon': 4.964584403492638e-07, 'beta_1': 0.9532848814911525, 'beta_2': 0.9314651556803287}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:18:58,666] Trial 193 finished with value: 0.12812499701976776 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0004179913142254212, 'rho': 0.9272863992541693}. Best is trial 172 with value: 0.675000011920929. [I 2021-03-10 20:19:17,990] Trial 196 finished with value: 0.690625011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000609593339702459, 'epsilon': 4.226899030767284e-06, 'beta_1': 0.9537281643103777, 'beta_2': 0.9390376626945334}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:19:38,813] Trial 197 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006007103568242361, 'epsilon': 1.8746142504733527e-07, 'beta_1': 0.9529451249376304, 'beta_2': 0.9382798604880763}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:19:39,310] Trial 198 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005539449247986231, 'epsilon': 2.7376156539040606e-07, 'beta_1': 0.9460945392794241, 'beta_2': 0.9390141051503318}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:19:39,783] Trial 199 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005207769454650343, 'epsilon': 1.9020653961874401e-07, 'beta_1': 0.9643669034287687, 'beta_2': 0.9392331540550217}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:19:51,089] Trial 200 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.000518692457492179, 'momentum': 0.011537279567684803, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:20:18,841] Trial 202 finished with value: 0.621874988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00033948835864462646, 'momentum': 0.021888049690766208, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:20:19,808] Trial 201 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005424936174349651, 'epsilon': 1.2648570176656076e-05, 'beta_1': 0.9577779864644533, 'beta_2': 0.9447213576923711}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:20:19,850] Trial 203 finished with value: 0.609375 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0003518313341026243, 'momentum': 0.006378817769298782, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:20:26,202] Trial 204 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008294306936653109, 'epsilon': 3.181022102233809e-06, 'beta_1': 0.9579616462714851, 'beta_2': 0.9248233016693512}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:21:00,279] Trial 205 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000773424002240256, 'epsilon': 4.168033748983158e-06, 'beta_1': 0.9577392936357951, 'beta_2': 0.9444338076184527}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:21:00,283] Trial 207 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007960475546296126, 'epsilon': 4.6765602964731935e-06, 'beta_1': 0.9474275953018759, 'beta_2': 0.9640414814657742}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:21:00,289] Trial 206 finished with value: 0.5843750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000881480684694114, 'epsilon': 5.168352925982067e-06, 'beta_1': 0.9474341430769481, 'beta_2': 0.9255327995171708}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:21:02,336] Trial 208 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010352818045268558, 'epsilon': 3.948535148329273e-06, 'beta_1': 0.9485473016689614, 'beta_2': 0.9436402626472703}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:21:40,621] Trial 211 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.001061751302748637, 'epsilon': 3.5128033638286334e-06, 'beta_1': 0.9370049715218621, 'beta_2': 0.9441291826678655}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:21:41,638] Trial 209 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010120516499347624, 'epsilon': 5.853117035271407e-06, 'beta_1': 0.9701806838499637, 'beta_2': 0.9768003651662085}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:21:41,646] Trial 210 finished with value: 0.6031249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.001022503096288786, 'epsilon': 6.7515781865894495e-06, 'beta_1': 0.9368741001146934, 'beta_2': 0.9775742777349807}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:21:42,066] Trial 212 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.001963560919568091, 'epsilon': 7.270250408612262e-06, 'beta_1': 0.9254094748197769, 'beta_2': 0.8659977479079779}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:22:19,136] Trial 213 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00026819653026466356, 'epsilon': 8.78066774189178e-07, 'beta_1': 0.95223387745046, 'beta_2': 0.9757768577368631}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:22:21,621] Trial 214 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006412854258510859, 'epsilon': 2.0715944388226716e-05, 'beta_1': 0.9553774793611891, 'beta_2': 0.9505341622559773}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:22:22,418] Trial 215 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006647000457116619, 'epsilon': 1.1950020829345363e-07, 'beta_1': 0.9555019471344806, 'beta_2': 0.8661854539843071}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:22:22,434] Trial 216 finished with value: 0.612500011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006755205202659696, 'epsilon': 9.499570794588806e-07, 'beta_1': 0.9546030690156044, 'beta_2': 0.950065891005591}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:22:58,414] Trial 217 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006394127433009097, 'epsilon': 9.893968985204617e-08, 'beta_1': 0.9551028626702472, 'beta_2': 0.9498861785912478}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:23:01,399] Trial 218 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000660944041764543, 'epsilon': 1.0108624685409879e-07, 'beta_1': 0.964391895626233, 'beta_2': 0.9329310414831752}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:23:02,642] Trial 219 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007473918216376704, 'epsilon': 2.4967677175554267e-06, 'beta_1': 0.9642077419088719, 'beta_2': 0.940775967850037}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:23:02,857] Trial 220 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00044525335818352467, 'epsilon': 4.020098178525099e-07, 'beta_1': 0.9620969519352783, 'beta_2': 0.9403844494237878}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:23:36,250] Trial 221 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00029948090461956987, 'epsilon': 4.1862922347682573e-07, 'beta_1': 0.9624004128357639, 'beta_2': 0.9327552566826378}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:23:41,251] Trial 222 finished with value: 0.668749988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004516311759167431, 'epsilon': 3.3610613759426167e-07, 'beta_1': 0.9597055752807823, 'beta_2': 0.9410458380819268}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:23:42,591] Trial 224 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003170113132075426, 'epsilon': 1.6265146769103345e-07, 'beta_1': 0.9755949551973128, 'beta_2': 0.9177100142464368}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:23:43,053] Trial 223 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003174476515984227, 'epsilon': 6.12588603235669e-07, 'beta_1': 0.9735938216788677, 'beta_2': 0.8610950237948602}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:24:10,028] Trial 225 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00017069657118766457, 'epsilon': 3.004860692788588e-07, 'beta_1': 0.9722798771118769, 'beta_2': 0.9377648294154913}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:24:22,009] Trial 226 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00036375032110871853, 'epsilon': 2.9564437774109926e-07, 'beta_1': 0.9507027154655645, 'beta_2': 0.9359305820873511}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:24:23,448] Trial 227 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004807997759381799, 'epsilon': 2.888621539991824e-07, 'beta_1': 0.9512162293192612, 'beta_2': 0.9372459324983078}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:24:23,511] Trial 228 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00046004141294539794, 'epsilon': 2.9532019198898956e-07, 'beta_1': 0.9515411559808915, 'beta_2': 0.9373433397998058}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:24:41,984] Trial 229 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00048687944830396, 'epsilon': 2.689646356699045e-07, 'beta_1': 0.9506043773672745, 'beta_2': 0.9454580510077031}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:25:03,962] Trial 230 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00048396380216734873, 'epsilon': 2.3570653986028028e-07, 'beta_1': 0.9593741519697623, 'beta_2': 0.8862922209356016}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:25:05,292] Trial 231 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00044086529354617557, 'epsilon': 7.098468326952726e-07, 'beta_1': 0.9588223558540664, 'beta_2': 0.9462665235195148}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:25:05,989] Trial 232 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0008315558266006928, 'rho': 0.8880669733482767, 'momentum': 0.05365317756393989, 'epsilon': 6.0905667314895e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:25:17,790] Trial 233 finished with value: 0.65625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003641910460883929, 'rho': 0.9000852439474433, 'momentum': 0.050095669012567734, 'epsilon': 6.524572880100659e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:25:46,376] Trial 236 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00037704769528586896, 'epsilon': 4.3954420395261357e-07, 'beta_1': 0.9443556171111506, 'beta_2': 0.9306227431282763}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:25:47,230] Trial 234 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003932641150144731, 'epsilon': 7.086498428557693e-07, 'beta_1': 0.9435316494101914, 'beta_2': 0.9420735851371103}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:25:48,262] Trial 235 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003921171860852702, 'epsilon': 5.627494838032861e-07, 'beta_1': 0.9439059661770806, 'beta_2': 0.9295345854165732}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:25:53,957] Trial 237 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005707382806472519, 'epsilon': 4.4486312508592036e-07, 'beta_1': 0.9443003984474156, 'beta_2': 0.9299234438413184}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:26:28,952] Trial 238 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005775163128392762, 'epsilon': 1.7778160292548353e-07, 'beta_1': 0.9858583705202658, 'beta_2': 0.9427547631368061}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:26:31,463] Trial 239 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002481627621008415, 'epsilon': 9.485676952494582e-06, 'beta_1': 0.9468681846875275, 'beta_2': 0.8910420076862516}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:26:31,953] Trial 240 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00024354124192113339, 'epsilon': 1.0578466653598173e-05, 'beta_1': 0.9681850929907083, 'beta_2': 0.8946202366286015}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:26:34,202] Trial 241 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00023330621050603263, 'epsilon': 2.0291120251730375e-07, 'beta_1': 0.9670918668959031, 'beta_2': 0.9558097263013445}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:27:10,950] Trial 242 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002444870515917911, 'epsilon': 9.276130085990596e-06, 'beta_1': 0.9671349237304635, 'beta_2': 0.956482556325427}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:27:15,583] Trial 243 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00022543934038776381, 'epsilon': 1.1107693884536344e-05, 'beta_1': 0.9469895191426396, 'beta_2': 0.8905582117666081}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:27:16,479] Trial 244 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002562219475772931, 'epsilon': 8.617428151526101e-06, 'beta_1': 0.9529283271592197, 'beta_2': 0.8919986214105347}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:27:16,671] Trial 245 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002750424987568978, 'epsilon': 7.97144811365224e-06, 'beta_1': 0.9479685144614987, 'beta_2': 0.9002054325422291}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:27:47,972] Trial 246 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007427466537869441, 'epsilon': 3.7089616886738327e-07, 'beta_1': 0.9466086749013553, 'beta_2': 0.9043905784610237}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:27:59,177] Trial 247 finished with value: 0.668749988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007687960301157691, 'epsilon': 1.5997979835420794e-05, 'beta_1': 0.9532930527533322, 'beta_2': 0.8575616917918505}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:28:00,178] Trial 249 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005060372904858615, 'rho': 0.9144555212822554, 'momentum': 0.00949759363100572, 'epsilon': 1.5334331702806704e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:28:00,288] Trial 248 finished with value: 0.65625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0007692906677491868, 'rho': 0.9148533934882943, 'momentum': 3.332918951071548e-05, 'epsilon': 3.4401477076415825e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:28:19,094] Trial 250 finished with value: 0.6031249761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004988718244787623, 'rho': 0.9041509142025298, 'momentum': 0.028271777380380517, 'epsilon': 4.025636191388953e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:28:38,652] Trial 251 finished with value: 0.625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0008346378886655772, 'rho': 0.9140176367100644, 'momentum': 0.0374488178562941, 'epsilon': 1.4169902768346661e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:28:41,069] Trial 252 finished with value: 0.625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006606255066578758, 'rho': 0.9272512585838911, 'momentum': 0.0773546483092241, 'epsilon': 4.706507123115326e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:28:41,070] Trial 253 finished with value: 0.612500011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008289709824534545, 'epsilon': 2.0514846081490527e-05, 'beta_1': 0.956772723223409, 'beta_2': 0.8716158931980343}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:28:52,383] Trial 254 finished with value: 0.6031249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007072411980181587, 'epsilon': 1.939530376354088e-05, 'beta_1': 0.9576969584038597, 'beta_2': 0.8503477385860561}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:29:17,257] Trial 255 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006253094888861124, 'epsilon': 3.067901464007942e-05, 'beta_1': 0.9561926831304719, 'beta_2': 0.8820880040369989}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:29:20,081] Trial 257 finished with value: 0.5874999761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00030841515742344394, 'momentum': 0.0022084242006671297, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:29:21,045] Trial 256 finished with value: 0.48750001192092896 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0003039916288833735, 'momentum': 1.9339461183286796e-05, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:29:27,354] Trial 258 finished with value: 0.6031249761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00032275418130482626, 'momentum': 6.417600513794822e-05, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:29:53,190] Trial 259 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0011613331102328104, 'momentum': 1.7710857204464327e-05, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:30:00,977] Trial 260 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005497247405071657, 'epsilon': 1.4818324790586356e-07, 'beta_1': 0.9525377473093591, 'beta_2': 0.8563096314949609}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:30:01,427] Trial 261 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.02333601098437403, 'epsilon': 1.593688597579806e-07, 'beta_1': 0.9606838796432373, 'beta_2': 0.8595894188178427}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:30:04,221] Trial 262 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0011929519873213313, 'epsilon': 2.2908873943281183e-06, 'beta_1': 0.9606340893184728, 'beta_2': 0.8546052280051399}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:30:25,652] Trial 263 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005563032864984641, 'epsilon': 2.2552102959340315e-06, 'beta_1': 0.9530342837285957, 'beta_2': 0.859582553523929}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:30:41,204] Trial 265 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006050335421873273, 'epsilon': 6.230140670317194e-06, 'beta_1': 0.9540098887103653, 'beta_2': 0.8554860731226385}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:30:41,380] Trial 264 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009641083349376834, 'epsilon': 1.1060522761992691e-06, 'beta_1': 0.9798838415878662, 'beta_2': 0.8579890136527392}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:30:43,532] Trial 266 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005549293024936354, 'epsilon': 2.0683408748614184e-07, 'beta_1': 0.9517911242001875, 'beta_2': 0.8556907445046145}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:30:58,730] Trial 267 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00042976861114542834, 'epsilon': 1.018866544481273e-06, 'beta_1': 0.9492280075252061, 'beta_2': 0.8642925342397024}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:31:23,268] Trial 268 finished with value: 0.515625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0016267480468804497, 'epsilon': 1.2493728260345472e-07, 'beta_1': 0.9799786853109727, 'beta_2': 0.8524913222563483}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:31:23,341] Trial 269 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00041982417598576243, 'epsilon': 1.3307818289535816e-07, 'beta_1': 0.9497424384107179, 'beta_2': 0.8641934665213077}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:31:24,185] Trial 270 finished with value: 0.653124988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00018917888467681393, 'rho': 0.8698681495082492, 'momentum': 0.0036791712899505724, 'epsilon': 1.0638134751089057e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:31:32,860] Trial 271 finished with value: 0.08124999701976776 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0004374452661884484, 'rho': 0.8773783355779586}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:32:05,225] Trial 274 finished with value: 0.453125 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0009257839710893821, 'rho': 0.893940631096143}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:32:05,922] Trial 273 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002142037530153152, 'rho': 0.8934587599643053, 'momentum': 0.013587140675018278, 'epsilon': 5.100812497721794e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:32:05,941] Trial 272 finished with value: 0.08124999701976776 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0009201024465274286, 'rho': 0.8775644512209755}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:32:08,432] Trial 275 finished with value: 0.659375011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0001488268901515362, 'rho': 0.8948908700420906, 'momentum': 1.2010153012198239e-05, 'epsilon': 7.017295815427496e-08}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:32:46,010] Trial 276 finished with value: 0.596875011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005005329966366014, 'epsilon': 5.807304533787873e-08, 'beta_1': 0.8869155386261969, 'beta_2': 0.9408249055638744}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:32:48,855] Trial 279 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005134740215328597, 'epsilon': 5.598028364954424e-08, 'beta_1': 0.893678545728135, 'beta_2': 0.911953477151847}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:32:49,299] Trial 278 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003605947855390858, 'epsilon': 7.382724859787812e-08, 'beta_1': 0.8926220895354418, 'beta_2': 0.9118141104383856}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:32:49,723] Trial 277 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00013850392251863077, 'epsilon': 5.724659432409004e-06, 'beta_1': 0.9382386615718635, 'beta_2': 0.9122256924406364}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:33:26,556] Trial 280 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003419496181339034, 'epsilon': 1.934110849511068e-07, 'beta_1': 0.9705558074217411, 'beta_2': 0.87566919409908}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:33:31,536] Trial 281 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00013878219864939067, 'epsilon': 9.621206569332122e-08, 'beta_1': 0.9395517082815014, 'beta_2': 0.8957373486284731}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:33:31,963] Trial 282 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00026424469300207306, 'epsilon': 3.271973099935232e-07, 'beta_1': 0.9394576291409467, 'beta_2': 0.8758927167601588}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:33:31,976] Trial 283 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00036335059905474984, 'epsilon': 3.1207913756802974e-07, 'beta_1': 0.9413262869882557, 'beta_2': 0.9869907712807984}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:34:03,476] Trial 284 finished with value: 0.643750011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0001447184370364911, 'rho': 0.8864709660099784, 'momentum': 1.1590281007967283e-05, 'epsilon': 3.238142451672039e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:34:15,703] Trial 285 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0001724140011272586, 'rho': 0.8869973281222481, 'momentum': 0.02032397622092028, 'epsilon': 3.6840353578890897e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:34:15,716] Trial 286 finished with value: 0.590624988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00040884493870782133, 'rho': 0.8856086945909307, 'momentum': 1.271997789089001e-05, 'epsilon': 2.2900040717482788e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:34:15,730] Trial 287 finished with value: 0.637499988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00019422737437286078, 'rho': 0.8861211858867498, 'momentum': 0.019477296542476697, 'epsilon': 1.3560003912188858e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:34:35,273] Trial 288 finished with value: 0.621874988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00045251181408803955, 'rho': 0.9069253697919525, 'momentum': 0.02113166925431188, 'epsilon': 3.1067537122939504e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:34:58,513] Trial 290 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005746696440515007, 'epsilon': 3.1575101160055494e-06, 'beta_1': 0.9464349882795273, 'beta_2': 0.8693981424853623}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:34:58,514] Trial 289 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004270695303909513, 'epsilon': 5.027269520870041e-07, 'beta_1': 0.9465737885657725, 'beta_2': 0.9473014948078681}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:34:58,533] Trial 291 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006973205172333657, 'epsilon': 1.5452714023144822e-06, 'beta_1': 0.9464190226070547, 'beta_2': 0.9481741123264735}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:35:08,710] Trial 292 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007773871746136262, 'epsilon': 4.7644739359383434e-07, 'beta_1': 0.9142413041973695, 'beta_2': 0.9647315136356363}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:35:39,994] Trial 293 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00026995271337321776, 'rho': 0.8730024661754519, 'momentum': 0.08831065816125665, 'epsilon': 2.35797685194829e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:35:39,996] Trial 294 finished with value: 0.574999988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0007210991185355998, 'rho': 0.8717024614995545, 'momentum': 8.994956157894984e-05, 'epsilon': 8.370260055561848e-08}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:35:40,517] Trial 295 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000282659405312669, 'epsilon': 3.629986389133292e-08, 'beta_1': 0.913600127717949, 'beta_2': 0.9444312765992646}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:35:45,491] Trial 296 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00011229105990947322, 'rho': 0.8970941939650077, 'momentum': 1.6289508876679198e-05, 'epsilon': 1.3827337349629712e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:36:22,384] Trial 297 finished with value: 0.621874988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00010743092955003001, 'rho': 0.8989711684654158, 'momentum': 2.3320436008340093e-05, 'epsilon': 8.156203381479427e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:36:22,732] Trial 298 finished with value: 0.625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 9.559044078860032e-05, 'rho': 0.8990665122772269, 'momentum': 1.7335812478963396e-05, 'epsilon': 4.1995877812798296e-08}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:36:22,752] Trial 299 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002232711435079626, 'rho': 0.8970366033106115, 'momentum': 3.1839499576602056e-05, 'epsilon': 1.129619176319178e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:36:23,166] Trial 300 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005682922317675604, 'epsilon': 1.5711846543856484e-07, 'beta_1': 0.9552752021554665, 'beta_2': 0.88824206420273}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:37:05,004] Trial 301 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005936488171643768, 'rho': 0.8907187696157499, 'momentum': 2.9440781534052738e-05, 'epsilon': 1.4795503550754495e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:37:05,351] Trial 304 finished with value: 0.5625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0013129383243620672, 'epsilon': 2.5226250347295587e-07, 'beta_1': 0.9646368781682508, 'beta_2': 0.9377329758071294}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:37:05,634] Trial 303 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005187310780344793, 'rho': 0.9109682268236958, 'momentum': 0.00018101833498329435, 'epsilon': 1.4980744903589175e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:37:05,678] Trial 302 finished with value: 0.640625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00021979817084591277, 'rho': 0.8647923086835981, 'momentum': 0.057845658198017486, 'epsilon': 1.4470275045281962e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:37:47,193] Trial 305 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003433280886298901, 'epsilon': 2.636854231816338e-07, 'beta_1': 0.9623692448856285, 'beta_2': 0.9338794090147908}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:37:48,410] Trial 306 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0001977641214712934, 'epsilon': 5.87149041727633e-06, 'beta_1': 0.9256237855676521, 'beta_2': 0.9596322095844586}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:37:48,808] Trial 307 finished with value: 0.653124988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003165418320905508, 'rho': 0.9103129345154763, 'momentum': 0.00019061162547462273, 'epsilon': 1.6773877929635818e-08}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:37:48,860] Trial 308 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00016535067204814322, 'rho': 0.9197707162916114, 'momentum': 0.00017330038494223443, 'epsilon': 4.418804164589589e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:38:29,795] Trial 309 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004744504211580559, 'epsilon': 6.859366196402243e-07, 'beta_1': 0.9502057694483901, 'beta_2': 0.9352438197536567}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:38:31,714] Trial 311 finished with value: 0.675000011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0016168966912816303, 'rho': 0.9053308670186742, 'momentum': 0.0004107904091365498, 'epsilon': 1.0191796058590397e-09}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:38:32,292] Trial 310 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004760910329694834, 'epsilon': 7.707474058786843e-07, 'beta_1': 0.9518966085916454, 'beta_2': 0.9259203526774797}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:38:32,638] Trial 312 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006713344598962332, 'rho': 0.9053746571983615, 'momentum': 0.000451186349775946, 'epsilon': 9.137706612923175e-08}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:39:12,304] Trial 313 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004893444614524498, 'epsilon': 7.72535483598341e-06, 'beta_1': 0.9583809049509704, 'beta_2': 0.9533702299048918}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:39:15,149] Trial 315 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0023124634419677786, 'rho': 0.9043435723723303, 'momentum': 0.000740623145813968, 'epsilon': 3.6239458798551374e-09}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:39:15,522] Trial 316 finished with value: 0.625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0003848409363262534, 'rho': 0.9026809849232826, 'momentum': 0.0006641303710133432, 'epsilon': 4.262039926053536e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:39:15,555] Trial 314 finished with value: 0.637499988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.002110760193975766, 'rho': 0.9043775085168456, 'momentum': 0.00037696144465910466, 'epsilon': 6.750177807317254e-09}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:39:54,983] Trial 317 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0016751035947120908, 'rho': 0.9315112931132881, 'momentum': 0.0003667669062135946, 'epsilon': 4.139045208889828e-09}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:39:58,279] Trial 320 finished with value: 0.637499988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.001628908359368024, 'rho': 0.9090243718375087, 'momentum': 0.0005459496625830441, 'epsilon': 2.6654574940244518e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:39:58,332] Trial 319 finished with value: 0.625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0018320837519333875, 'rho': 0.9123194617306648, 'momentum': 0.0012097153158029004, 'epsilon': 2.8840208600614014e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:39:58,768] Trial 318 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000814889252528727, 'rho': 0.9024266984073607, 'momentum': 0.0003451729319607792, 'epsilon': 3.183712422420379e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:40:36,097] Trial 321 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0009763843396426321, 'rho': 0.9083695992843327, 'momentum': 0.00024161854664799564, 'epsilon': 3.999337921558124e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:40:40,634] Trial 324 finished with value: 0.574999988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00038110115657419074, 'momentum': 0.0017208964989295322, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:40:42,229] Trial 323 finished with value: 0.503125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.003203765790264962, 'rho': 0.8809825408764052, 'momentum': 0.0018676272786534206, 'epsilon': 5.799446330444893e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:40:42,366] Trial 322 finished with value: 0.559374988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0008259474448844499, 'rho': 0.8920326467659668, 'momentum': 0.0011932834333838807, 'epsilon': 5.044132554748364e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:41:09,951] Trial 325 finished with value: 0.640625 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.003445186695952227, 'momentum': 0.00229439821419382, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:41:23,145] Trial 328 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006767719033442722, 'epsilon': 2.266989796169404e-07, 'beta_1': 0.9608341038370001, 'beta_2': 0.9515388174691867}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:41:23,148] Trial 326 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006241963593865299, 'epsilon': 1.1753806233795182e-09, 'beta_1': 0.8770191320830305, 'beta_2': 0.952516171188783}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:41:23,189] Trial 327 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006478599368600047, 'epsilon': 1.067862624790656e-09, 'beta_1': 0.8754626670073632, 'beta_2': 0.9517272828951324}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:41:40,218] Trial 329 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000537832393341902, 'epsilon': 3.393074588603366e-07, 'beta_1': 0.9300623857294812, 'beta_2': 0.9697314208105396}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:42:03,938] Trial 331 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000549516618986723, 'epsilon': 1.2483135677393896e-06, 'beta_1': 0.9434147317602916, 'beta_2': 0.9684868138275785}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:42:05,265] Trial 330 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005482926176034282, 'epsilon': 1.038225989003624e-06, 'beta_1': 0.9361390397933187, 'beta_2': 0.9680861864680486}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:42:05,291] Trial 332 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0002677718389500162, 'rho': 0.9247744924953598, 'momentum': 0.00025137138340792424, 'epsilon': 1.41936577430371e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:42:14,546] Trial 333 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00028843948613023186, 'epsilon': 2.025089344039989e-06, 'beta_1': 0.9439110883605454, 'beta_2': 0.9646877020032204}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:42:44,995] Trial 334 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0014543044350371474, 'epsilon': 3.430079850724613e-06, 'beta_1': 0.9290419757301239, 'beta_2': 0.9708316337894038}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:42:47,417] Trial 335 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00026656046582801917, 'epsilon': 3.739099257116066e-07, 'beta_1': 0.9210101470108752, 'beta_2': 0.9738926219706152}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:42:48,015] Trial 336 finished with value: 0.640625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00030937983957627204, 'rho': 0.9207401167449258, 'momentum': 1.0448770095649082e-05, 'epsilon': 3.9624596858366653e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:42:51,603] Trial 337 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004445720616739128, 'epsilon': 4.2396099351246054e-07, 'beta_1': 0.9489228860466167, 'beta_2': 0.8613304665775964}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:43:25,027] Trial 338 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0012014228601297228, 'epsilon': 4.308974926353178e-07, 'beta_1': 0.976253366738524, 'beta_2': 0.9584296753706701}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:43:31,954] Trial 339 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004150620742368267, 'epsilon': 7.607594809399457e-07, 'beta_1': 0.929022377796024, 'beta_2': 0.9585427880722565}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:43:32,004] Trial 340 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00016927342976900422, 'rho': 0.9154288177212307, 'momentum': 0.0005513052314449974, 'epsilon': 1.0883246498123217e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:43:32,481] Trial 341 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0012493088015817252, 'epsilon': 3.1043689080943555e-07, 'beta_1': 0.9549857145607389, 'beta_2': 0.9403674379516074}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:43:56,820] Trial 342 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004945997007806698, 'epsilon': 1.8590281956642316e-07, 'beta_1': 0.9537240394209974, 'beta_2': 0.9408931052053522}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:44:12,484] Trial 344 finished with value: 0.6000000238418579 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0005313334881972702, 'momentum': 0.0008180425547158241, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:44:14,040] Trial 345 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00048560112069410575, 'epsilon': 6.017041897740938e-07, 'beta_1': 0.9520752129747759, 'beta_2': 0.8816571500170948}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:44:14,093] Trial 343 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005091520822587793, 'epsilon': 2.884190649818166e-07, 'beta_1': 0.9347921720772243, 'beta_2': 0.9001055854188772}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:44:28,854] Trial 346 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007387912174399276, 'epsilon': 1.5075230074686205e-05, 'beta_1': 0.9877139344433387, 'beta_2': 0.9449089255568736}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:44:54,498] Trial 349 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000361572238383067, 'epsilon': 8.573873229925886e-07, 'beta_1': 0.9416906241665159, 'beta_2': 0.93853972164028}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:44:56,072] Trial 347 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003617880846344901, 'epsilon': 6.584949326004222e-07, 'beta_1': 0.935536916020777, 'beta_2': 0.8510401977595273}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:44:57,020] Trial 348 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00036976169277000117, 'epsilon': 3.449285516786205e-07, 'beta_1': 0.9414640736883466, 'beta_2': 0.9804035075709662}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:45:04,178] Trial 350 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003646646272635483, 'epsilon': 1.2694599907236169e-05, 'beta_1': 0.9568513882132874, 'beta_2': 0.8502191308550833}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:45:39,148] Trial 352 finished with value: 0.606249988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008920038074255363, 'epsilon': 1.4237595445909694e-05, 'beta_1': 0.9888677990447261, 'beta_2': 0.9814383581087283}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:45:39,394] Trial 353 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006030065481978185, 'rho': 0.9353800254304322, 'momentum': 0.040092632142868394, 'epsilon': 9.93357443015585e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:45:39,729] Trial 351 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008356227453662174, 'epsilon': 1.688611226328601e-05, 'beta_1': 0.9881492970555622, 'beta_2': 0.9823623065292219}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:45:41,668] Trial 354 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010677704865788363, 'epsilon': 9.421786705395764e-06, 'beta_1': 0.9862196977527098, 'beta_2': 0.8858035804555011}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:46:23,954] Trial 355 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007152117232258979, 'epsilon': 1.8094566420862365e-05, 'beta_1': 0.958735310050852, 'beta_2': 0.9457083352873433}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:46:25,491] Trial 358 finished with value: 0.668749988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006001693987819751, 'rho': 0.9432372537841848, 'momentum': 0.027000942485794037, 'epsilon': 2.102030603534212e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:46:26,462] Trial 356 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0001331703606985666, 'rho': 0.9346757317953043, 'momentum': 0.03855701893938258, 'epsilon': 1.835976079478395e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:46:26,484] Trial 357 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0001988618012664173, 'rho': 0.9597748292108298, 'momentum': 0.03637793928538323, 'epsilon': 1.726513831702738e-08}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:47:09,140] Trial 359 finished with value: 0.65625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00014021023463470578, 'rho': 0.9549189649289321, 'momentum': 0.03554388253511228, 'epsilon': 2.095744908227309e-08}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:47:11,613] Trial 362 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00023392180244860217, 'rho': 0.9474450104024222, 'momentum': 0.02836504245129089, 'epsilon': 2.1036226535589966e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:47:11,697] Trial 361 finished with value: 0.65625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005742296419539732, 'rho': 0.9469091104852272, 'momentum': 0.027417922030117947, 'epsilon': 5.825758526365795e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:47:12,099] Trial 360 finished with value: 0.609375 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.000595297297830839, 'rho': 0.9563418690303457, 'momentum': 0.0400227706582335, 'epsilon': 1.9999282328457667e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:47:52,137] Trial 363 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00023623154233126396, 'rho': 0.9470469552269799, 'momentum': 0.027109485700547064, 'epsilon': 8.00623213474409e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:47:54,668] Trial 366 finished with value: 0.621874988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006877291845691386, 'rho': 0.9702284917934361, 'momentum': 0.07791383736191303, 'epsilon': 1.1931785479317099e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:47:55,288] Trial 364 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006210534030950536, 'rho': 0.9389783509783655, 'momentum': 0.026911424735542065, 'epsilon': 2.7287590973400748e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:47:55,311] Trial 365 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006335433086335635, 'rho': 0.9394201450629818, 'momentum': 0.009797033745499685, 'epsilon': 9.86656031803114e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:48:32,468] Trial 367 finished with value: 0.559374988079071 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.008379349982118858, 'rho': 0.9393191957811106}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:48:35,482] Trial 369 finished with value: 0.643750011920929 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0007532065238159195, 'momentum': 0.006380040221729927, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:48:37,046] Trial 368 finished with value: 0.03437500074505806 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0007041483187318186, 'rho': 0.9397492797894755}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:48:37,678] Trial 370 finished with value: 0.24687500298023224 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0004474731551218515, 'rho': 0.94188523478726}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:49:12,106] Trial 371 finished with value: 0.6000000238418579 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00043224820476687434, 'momentum': 0.016303423771830784, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:49:19,503] Trial 372 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00041435805495510447, 'rho': 0.917246891533331, 'momentum': 0.0536425096235848, 'epsilon': 4.500935028175933e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:49:19,597] Trial 373 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0004329928968218592, 'rho': 0.9171280980078561, 'momentum': 0.013230328405696509, 'epsilon': 5.424258700926837e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:49:19,605] Trial 374 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0009936856867858264, 'rho': 0.9295584316950923, 'momentum': 0.01379448060023057, 'epsilon': 6.201931617209219e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:49:44,366] Trial 375 finished with value: 0.574999988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0009694163882059561, 'rho': 0.9505936705633296, 'momentum': 0.05882449657777692, 'epsilon': 3.663916801025658e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:50:01,627] Trial 378 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00030718209741434463, 'rho': 0.912210224504569, 'momentum': 0.06781964271611034, 'epsilon': 1.5226458914355137e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:50:01,668] Trial 376 finished with value: 0.65625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005454590052372052, 'rho': 0.9296638679116752, 'momentum': 0.0700176316543075, 'epsilon': 2.0198328496046613e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:50:02,189] Trial 377 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005587567413857987, 'rho': 0.9358597017546887, 'momentum': 0.0010002516560361704, 'epsilon': 2.4870084074512245e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:50:14,536] Trial 379 finished with value: 0.637499988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00030418351450323036, 'rho': 0.9257983047020762, 'momentum': 0.0014938030527459224, 'epsilon': 1.2825893394615353e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:50:42,412] Trial 381 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007888101345620524, 'epsilon': 1.0369254376751984e-06, 'beta_1': 0.9051378036801895, 'beta_2': 0.9626294159717601}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:50:42,420] Trial 382 finished with value: 0.671875 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008324587522244486, 'epsilon': 4.877207945524152e-07, 'beta_1': 0.9709943337515194, 'beta_2': 0.9614075351039448}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:50:42,576] Trial 380 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000839205865169132, 'epsilon': 2.432352954230013e-09, 'beta_1': 0.9316441967090474, 'beta_2': 0.94947070203054}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:50:49,674] Trial 383 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008456997142988766, 'epsilon': 2.4183937631906997e-07, 'beta_1': 0.9646281600056396, 'beta_2': 0.907148146072839}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:51:24,465] Trial 386 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005471924567569827, 'epsilon': 5.442893311712407e-07, 'beta_1': 0.9703289624113277, 'beta_2': 0.9660043971306553}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:51:24,467] Trial 385 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008743700053887091, 'epsilon': 5.327043187624008e-07, 'beta_1': 0.9319621719279634, 'beta_2': 0.9717742835387314}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:51:24,505] Trial 384 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007690317062721919, 'epsilon': 2.4460316388715585e-09, 'beta_1': 0.9321970165474354, 'beta_2': 0.955291961042315}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:51:26,272] Trial 387 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006069147269129831, 'epsilon': 5.327850256880292e-07, 'beta_1': 0.9222644014606732, 'beta_2': 0.9653536078184775}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:52:07,689] Trial 388 finished with value: 0.606249988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006488901268592838, 'epsilon': 3.740525802254706e-07, 'beta_1': 0.948439622053105, 'beta_2': 0.9595265687181737}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:52:09,095] Trial 389 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004849794321956138, 'epsilon': 3.944449442107214e-07, 'beta_1': 0.9660813543426341, 'beta_2': 0.9611729266793835}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:52:09,111] Trial 390 finished with value: 0.653124988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006706379175059576, 'rho': 0.8795546709571722, 'momentum': 0.0448504608118767, 'epsilon': 3.27573339038353e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:52:09,988] Trial 391 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005077164706836758, 'rho': 0.8905575638215986, 'momentum': 0.04728152368807272, 'epsilon': 7.757107669674368e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:52:50,618] Trial 392 finished with value: 0.628125011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00026605307450104377, 'rho': 0.8900748269631699, 'momentum': 0.00015543239203059595, 'epsilon': 0.0046608760760451715}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:52:52,465] Trial 394 finished with value: 0.596875011920929 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00019647550723926642, 'momentum': 0.004856228799204125, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:52:52,491] Trial 395 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0002501775638352263, 'momentum': 0.004684885022194127, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:52:52,566] Trial 393 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00019144897634808331, 'rho': 0.8898330803423296, 'momentum': 0.09733001418785628, 'epsilon': 8.976983055444493e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:53:35,337] Trial 396 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003352336046671744, 'epsilon': 1.128565140146069e-05, 'beta_1': 0.9456396698166155, 'beta_2': 0.9493936306643963}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:53:35,695] Trial 399 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035507940328181084, 'epsilon': 2.687103473284196e-07, 'beta_1': 0.9450810636498309, 'beta_2': 0.9538814706516211}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:53:35,738] Trial 397 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003462450919858387, 'epsilon': 8.990140669180875e-06, 'beta_1': 0.9463721795043374, 'beta_2': 0.9557982273408285}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:53:35,856] Trial 398 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00040378608847043963, 'epsilon': 9.849866696882872e-09, 'beta_1': 0.9458756667025657, 'beta_2': 0.9553991524704092}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:54:15,812] Trial 400 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003912167186431201, 'epsilon': 7.778865926380155e-06, 'beta_1': 0.943777698033721, 'beta_2': 0.954031210820844}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:54:18,602] Trial 403 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003152161870747163, 'epsilon': 1.3736918503346375e-06, 'beta_1': 0.9389515571360842, 'beta_2': 0.9619340394203786}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:54:18,624] Trial 402 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003031750869550072, 'epsilon': 1.3357712305468355e-05, 'beta_1': 0.9826720962906716, 'beta_2': 0.9486139288807051}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:54:18,722] Trial 401 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003188091158499748, 'epsilon': 2.330579134939487e-05, 'beta_1': 0.9384811550523097, 'beta_2': 0.9492370931789245}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:54:55,018] Trial 404 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00042752582417106684, 'epsilon': 1.1888894135282794e-05, 'beta_1': 0.9386467662872422, 'beta_2': 0.9490149767061231}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:54:59,741] Trial 406 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004259524795076866, 'epsilon': 1.1907935072087372e-08, 'beta_1': 0.9500982812327683, 'beta_2': 0.9454545658385686}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:54:59,823] Trial 407 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003945926288845481, 'epsilon': 7.005655816548188e-07, 'beta_1': 0.9495146369170799, 'beta_2': 0.9572117618570658}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:55:00,263] Trial 405 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00045768983378322167, 'epsilon': 5.350166128256543e-09, 'beta_1': 0.9514298021897062, 'beta_2': 0.957116632993135}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:55:30,272] Trial 408 finished with value: 0.6031249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 2.97398979288849e-05, 'epsilon': 7.185004525574525e-07, 'beta_1': 0.9539788199325261, 'beta_2': 0.9572783725440441}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:55:40,050] Trial 410 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00024370665172204314, 'epsilon': 5.3635088278183725e-09, 'beta_1': 0.9546137222583523, 'beta_2': 0.9629404670271842}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:55:40,163] Trial 409 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000532807469167345, 'epsilon': 5.76006154674659e-09, 'beta_1': 0.9531317459637046, 'beta_2': 0.8567478716404741}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:55:41,546] Trial 411 finished with value: 0.640625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005204990184881345, 'rho': 0.9222670869130635, 'momentum': 9.296367313428981e-05, 'epsilon': 4.6041282207609177e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:56:02,141] Trial 412 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005433724195896138, 'epsilon': 9.370684925495232e-09, 'beta_1': 0.9421779426600194, 'beta_2': 0.9693899051352647}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:56:22,577] Trial 413 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010973627190562722, 'epsilon': 7.918756656082394e-06, 'beta_1': 0.9624093158318653, 'beta_2': 0.9435568829664623}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:56:22,661] Trial 414 finished with value: 0.581250011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006237485319241971, 'epsilon': 4.081721257971125e-07, 'beta_1': 0.9472218541151475, 'beta_2': 0.9431094405347422}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:56:22,692] Trial 415 finished with value: 0.643750011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0011224171347901065, 'rho': 0.9003247379352103, 'momentum': 0.02368374835755558, 'epsilon': 1.6375853672161764e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:56:35,831] Trial 416 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003553724494845514, 'epsilon': 2.8834350710223763e-08, 'beta_1': 0.9614720770543407, 'beta_2': 0.9428994372479822}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:57:05,386] Trial 418 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00021574100248512973, 'epsilon': 1.5831348519934817e-05, 'beta_1': 0.9734375941916474, 'beta_2': 0.8685792145344188}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:57:06,701] Trial 417 finished with value: 0.596875011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006710114020908138, 'rho': 0.9005108482774035, 'momentum': 0.008109757045911919, 'epsilon': 1.9058962272137555e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:57:06,707] Trial 419 finished with value: 0.612500011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0007365112812366895, 'rho': 0.8837292514743874, 'momentum': 0.0025149319277386942, 'epsilon': 1.823129453528168e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:57:11,475] Trial 420 finished with value: 0.612500011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0007819987416240922, 'rho': 0.9077203346907985, 'momentum': 0.03244113755484953, 'epsilon': 1.9996533121112285e-05}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:57:48,492] Trial 421 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00046054871067736274, 'epsilon': 2.525435883932219e-07, 'beta_1': 0.9578224963780492, 'beta_2': 0.9512761085930046}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:57:50,419] Trial 423 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000943458493764387, 'epsilon': 2.525173244729378e-07, 'beta_1': 0.9573451902526255, 'beta_2': 0.9529627346369922}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:57:50,936] Trial 422 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.002490890534546596, 'epsilon': 2.569736933007105e-07, 'beta_1': 0.9567894241506713, 'beta_2': 0.9506228702234923}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:57:51,638] Trial 424 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00046834162822621806, 'epsilon': 1.3871496967263221e-09, 'beta_1': 0.9450399953710067, 'beta_2': 0.8536511271808033}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:58:30,389] Trial 425 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005869728082818886, 'epsilon': 1.3103602992323305e-07, 'beta_1': 0.9490619975778352, 'beta_2': 0.9743332603986448}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:58:31,158] Trial 427 finished with value: 0.637499988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.000608651977490941, 'momentum': 0.022147182959295554, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:58:32,352] Trial 426 finished with value: 0.675000011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002766445598726118, 'epsilon': 1.8361709808245998e-06, 'beta_1': 0.9453044862651125, 'beta_2': 0.8532823279885098}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:58:32,737] Trial 428 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005855930842885214, 'epsilon': 1.3042916453092395e-06, 'beta_1': 0.9484995941879741, 'beta_2': 0.9615524167433872}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:59:11,650] Trial 429 finished with value: 0.612500011920929 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0005997568134569629, 'momentum': 0.010950484679383336, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:59:13,039] Trial 430 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007197497862277446, 'epsilon': 1.0405415419341509e-05, 'beta_1': 0.9479785107553587, 'beta_2': 0.9678051819443774}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:59:13,319] Trial 432 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007531670579111555, 'epsilon': 9.275700055302772e-07, 'beta_1': 0.9424353337591942, 'beta_2': 0.9769161301401246}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:59:13,346] Trial 431 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007332214649686189, 'epsilon': 6.234323087141309e-07, 'beta_1': 0.9492028041868938, 'beta_2': 0.9681196696722562}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:59:53,227] Trial 433 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007678820892253728, 'epsilon': 9.031597502573595e-07, 'beta_1': 0.942300708733168, 'beta_2': 0.974875659668362}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:59:55,685] Trial 434 finished with value: 0.612500011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000297199847488159, 'epsilon': 1.7632201979867352e-06, 'beta_1': 0.9411939620879002, 'beta_2': 0.9738462151117964}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:59:56,751] Trial 435 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00025571488767653117, 'epsilon': 1.7683908122550393e-06, 'beta_1': 0.9514035413389998, 'beta_2': 0.9730923596640166}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 20:59:56,805] Trial 436 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000962785936523444, 'epsilon': 1.6997303345078381e-06, 'beta_1': 0.9517280162506132, 'beta_2': 0.97434749539042}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:00:35,446] Trial 437 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00027284152383868284, 'epsilon': 2.6399517492840905e-06, 'beta_1': 0.9456512942364403, 'beta_2': 0.8571346904222722}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:00:40,715] Trial 440 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0013466717236791427, 'epsilon': 5.272851995995322e-07, 'beta_1': 0.9544018332937652, 'beta_2': 0.9662823829385797}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:00:41,149] Trial 438 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0009475485294172665, 'rho': 0.8967465083682443, 'momentum': 5.308381177641277e-05, 'epsilon': 1.0029646211466557e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:00:41,313] Trial 439 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035571623039977094, 'epsilon': 4.783914981125985e-07, 'beta_1': 0.8558703831757862, 'beta_2': 0.9705953800641907}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:01:14,055] Trial 441 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0014426670610128275, 'rho': 0.9327132481492142, 'momentum': 7.507998771893066e-05, 'epsilon': 1.7076304058193426e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:01:24,078] Trial 443 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00017954411056239348, 'epsilon': 3.194547754602467e-07, 'beta_1': 0.9351093713359594, 'beta_2': 0.8615503028142486}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:01:24,624] Trial 442 finished with value: 0.668749988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00042682352646283817, 'epsilon': 9.83260159961589e-07, 'beta_1': 0.945479616751967, 'beta_2': 0.9459468885561361}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:01:25,692] Trial 444 finished with value: 0.06562499701976776 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.00015960421106035004, 'rho': 0.9449471259650821}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:01:45,758] Trial 445 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0005185532104404722, 'rho': 0.9134865823673756, 'momentum': 0.00046659393864798113, 'epsilon': 1.521551702000983e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:02:06,332] Trial 447 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005110878685292894, 'epsilon': 1.0924514595720449e-06, 'beta_1': 0.9374795792812279, 'beta_2': 0.9468128365063455}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:02:07,666] Trial 448 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000542785031318151, 'epsilon': 9.68191540746906e-07, 'beta_1': 0.9484789373988947, 'beta_2': 0.9884882032840944}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:02:07,686] Trial 446 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003993599783516519, 'epsilon': 9.12485002147556e-05, 'beta_1': 0.9449351962431173, 'beta_2': 0.8532361898128026}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:02:17,826] Trial 449 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00040099201211666805, 'epsilon': 1.3090602994282715e-06, 'beta_1': 0.9437657229579431, 'beta_2': 0.9467406807125903}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:02:49,146] Trial 452 finished with value: 0.34687501192092896 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.04706183627270002, 'epsilon': 2.1868379329121388e-06, 'beta_1': 0.9559362219907576, 'beta_2': 0.9789823753885568}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:02:49,169] Trial 450 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006808102520571992, 'epsilon': 0.00011102278826095859, 'beta_1': 0.949494418838277, 'beta_2': 0.9461233649743767}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:02:49,178] Trial 451 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004280848746567388, 'epsilon': 1.3336327240098092e-06, 'beta_1': 0.9683749762729819, 'beta_2': 0.947154235383938}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:02:54,559] Trial 453 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003278478856658116, 'epsilon': 3.672163429290543e-07, 'beta_1': 0.9521886685225773, 'beta_2': 0.8592541092278457}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:03:30,756] Trial 456 finished with value: 0.581250011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.002706888243824481, 'epsilon': 7.102874606558566e-07, 'beta_1': 0.9535430696863318, 'beta_2': 0.9672820308088473}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:03:30,820] Trial 454 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00023159534618505932, 'epsilon': 6.070041556438169e-07, 'beta_1': 0.9529738220423082, 'beta_2': 0.964838556871793}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:03:32,425] Trial 455 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008658747613178686, 'epsilon': 6.701579090140892e-07, 'beta_1': 0.9525359284468327, 'beta_2': 0.9702211037328713}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:03:33,869] Trial 457 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00022280902707006258, 'epsilon': 6.488105228961775e-07, 'beta_1': 0.9405125021995201, 'beta_2': 0.9599262245090484}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:04:15,545] Trial 460 finished with value: 0.5687500238418579 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00030846975858909406, 'momentum': 0.00012415088767243052, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:04:15,559] Trial 459 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010872764860672475, 'epsilon': 8.314456623916563e-07, 'beta_1': 0.9486153025103956, 'beta_2': 0.9691797886392401}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:04:15,600] Trial 458 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008482425569125843, 'epsilon': 0.0005050927676170707, 'beta_1': 0.9478667301705797, 'beta_2': 0.9706838619131544}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:04:15,630] Trial 461 finished with value: 0.640625 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0006196919483950968, 'rho': 0.9060085537520317, 'momentum': 0.08220908010486573, 'epsilon': 1.3249266076363938e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:04:57,049] Trial 465 finished with value: 0.668749988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006486963671567854, 'epsilon': 4.718121372491509e-07, 'beta_1': 0.9585969146750067, 'beta_2': 0.9850336892507533}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:04:58,551] Trial 462 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00047960463902995136, 'epsilon': 5.896730431223702e-06, 'beta_1': 0.9598509449975532, 'beta_2': 0.9627263877768214}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:04:58,562] Trial 463 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00045886077389301874, 'epsilon': 3.6236999266145703e-05, 'beta_1': 0.958837008700339, 'beta_2': 0.9644569290996151}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:04:58,813] Trial 464 finished with value: 0.668749988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00062480207849221, 'epsilon': 5.42738807950244e-07, 'beta_1': 0.9448197654346927, 'beta_2': 0.963375884418918}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:05:37,424] Trial 469 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006169388770466375, 'epsilon': 3.986999304261025e-07, 'beta_1': 0.9452899261850047, 'beta_2': 0.9516494566739335}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:05:39,863] Trial 466 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007073969348899852, 'epsilon': 4.817913057168886e-07, 'beta_1': 0.9589422254258372, 'beta_2': 0.9846923304564946}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:05:39,890] Trial 468 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006430491318749204, 'epsilon': 5.246168923481657e-05, 'beta_1': 0.9598746121762177, 'beta_2': 0.9518911087937746}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:05:39,975] Trial 467 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006169764792923723, 'epsilon': 4.2286632139788466e-05, 'beta_1': 0.963910585420566, 'beta_2': 0.9828875544433536}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:06:17,181] Trial 470 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006904532378626822, 'epsilon': 4.627891234984744e-05, 'beta_1': 0.956354779110572, 'beta_2': 0.9655838121745322}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:06:20,025] Trial 471 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00059774405415514, 'epsilon': 5.128566503786554e-07, 'beta_1': 0.9457619764966892, 'beta_2': 0.984772843572308}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:06:20,078] Trial 473 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008978052296667173, 'epsilon': 1.1985444850188132e-06, 'beta_1': 0.9548765905028586, 'beta_2': 0.9417029178237785}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:06:20,293] Trial 472 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008602244775457138, 'epsilon': 1.1554403101897128e-06, 'beta_1': 0.9430952418738229, 'beta_2': 0.9583559744158241}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:06:57,930] Trial 474 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008274935605971841, 'epsilon': 5.706837664079991e-07, 'beta_1': 0.9501102430541962, 'beta_2': 0.9598321914237333}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:07:01,258] Trial 477 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00041157944647918096, 'epsilon': 3.824068104828186e-07, 'beta_1': 0.93987433635934, 'beta_2': 0.9535589192131385}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:07:01,399] Trial 475 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004841822263596053, 'epsilon': 6.656177866718278e-05, 'beta_1': 0.939180246714709, 'beta_2': 0.9593472850341048}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:07:01,478] Trial 476 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008163679510917181, 'epsilon': 8.615075169755375e-07, 'beta_1': 0.9397345742327398, 'beta_2': 0.9893994223704542}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:07:38,247] Trial 478 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000535678733540355, 'epsilon': 4.0431376355319827e-07, 'beta_1': 0.9370986739138123, 'beta_2': 0.9544283220271924}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:07:43,558] Trial 479 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005065883516896561, 'epsilon': 8.599786813855034e-07, 'beta_1': 0.8797114276908898, 'beta_2': 0.9400013599100303}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:07:43,985] Trial 480 finished with value: 0.6000000238418579 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0005439792450598649, 'momentum': 0.016079022461603778, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:07:44,043] Trial 481 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0010802324704401818, 'momentum': 0.005573251640527712, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:08:17,576] Trial 482 finished with value: 0.596875011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0012196480321960091, 'epsilon': 3.20024698892026e-07, 'beta_1': 0.9274917171520927, 'beta_2': 0.8511181229684788}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:08:27,972] Trial 485 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004545371676644984, 'epsilon': 2.3918595411300962e-05, 'beta_1': 0.9556933622310314, 'beta_2': 0.8503223505515611}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:08:28,004] Trial 483 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010345987808672223, 'epsilon': 2.725831292887423e-07, 'beta_1': 0.9784978806784639, 'beta_2': 0.8515222176154313}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:08:28,125] Trial 484 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010575398699468439, 'epsilon': 3.247720017068814e-07, 'beta_1': 0.9624774929116933, 'beta_2': 0.9641731395432726}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:08:49,942] Trial 486 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00045009243045638877, 'epsilon': 1.607884563680469e-06, 'beta_1': 0.9571917057463718, 'beta_2': 0.9627297825366457}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:09:11,617] Trial 488 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003866061531326301, 'epsilon': 6.076339732710549e-07, 'beta_1': 0.9470935629913962, 'beta_2': 0.9427239181649293}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:09:11,623] Trial 487 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007472544329317103, 'epsilon': 7.013108485874684e-07, 'beta_1': 0.9516636125634894, 'beta_2': 0.9385546126072094}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:09:11,628] Trial 489 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003818283869083847, 'epsilon': 1.81848492105808e-06, 'beta_1': 0.950760899086276, 'beta_2': 0.9444676546878812}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:09:22,008] Trial 490 finished with value: 0.19687500596046448 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0006935582070587342, 'rho': 0.9827538032581444}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:09:53,031] Trial 492 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005886273873002051, 'epsilon': 2.034733596094363e-07, 'beta_1': 0.9441537766577646, 'beta_2': 0.9491508553110071}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:09:54,158] Trial 491 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005556619392092293, 'epsilon': 0.0017585729471744575, 'beta_1': 0.9344951685523556, 'beta_2': 0.9485100134511198}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:09:54,225] Trial 493 finished with value: 0.4625000059604645 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0005794430257570502, 'rho': 0.9897758283169198}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:09:58,233] Trial 494 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0019684235388235847, 'epsilon': 4.4089380086370864e-07, 'beta_1': 0.9430432812226109, 'beta_2': 0.8642162814472915}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:10:35,088] Trial 496 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00032419013244558106, 'epsilon': 4.519606513200322e-07, 'beta_1': 0.9469670672517363, 'beta_2': 0.8639002195376593}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:10:35,642] Trial 497 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004723414662386436, 'epsilon': 4.03178661877808e-05, 'beta_1': 0.9599663954014116, 'beta_2': 0.9783433730734493}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:10:36,065] Trial 495 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035102263844046357, 'epsilon': 1.9007157148711444e-09, 'beta_1': 0.9468654998346026, 'beta_2': 0.9567469210178824}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:10:36,938] Trial 498 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004176437794267974, 'epsilon': 2.7588721595949322e-05, 'beta_1': 0.9662990064838599, 'beta_2': 0.8542588034640934}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:11:16,598] Trial 500 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003854282023454083, 'epsilon': 1.1209175424552188e-06, 'beta_1': 0.9462265078199914, 'beta_2': 0.9566659049945797}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:11:16,682] Trial 499 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007026690835028184, 'epsilon': 9.698216735659736e-07, 'beta_1': 0.965652219546777, 'beta_2': 0.9779437666790354}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:11:17,339] Trial 501 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007421487813526756, 'epsilon': 2.2434722355792793e-07, 'beta_1': 0.9232269185608609, 'beta_2': 0.8536702598408172}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:11:17,474] Trial 502 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007500415873964229, 'epsilon': 1.3219709018455912e-08, 'beta_1': 0.9499301824181862, 'beta_2': 0.9551599535892413}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:11:58,269] Trial 506 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000527453339244093, 'epsilon': 1.9803520732496964e-07, 'beta_1': 0.9523151097983735, 'beta_2': 0.8577673258161993}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:11:58,853] Trial 504 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006739459283472083, 'epsilon': 0.0003062307972664696, 'beta_1': 0.9233582030675853, 'beta_2': 0.9482506558648854}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:11:59,111] Trial 503 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00027611637728701795, 'epsilon': 5.404848796637669e-07, 'beta_1': 0.9502900397384964, 'beta_2': 0.8583818036595737}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:11:59,231] Trial 505 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004998232259683954, 'epsilon': 5.777339596346458e-07, 'beta_1': 0.9493840206806438, 'beta_2': 0.9542975921002647}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:12:37,764] Trial 507 finished with value: 0.675000011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00047748645557741455, 'epsilon': 3.1479265203460403e-05, 'beta_1': 0.9550695180313098, 'beta_2': 0.9721256021360812}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:12:39,505] Trial 509 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009024387677599286, 'epsilon': 7.206340451608104e-07, 'beta_1': 0.9553649683983706, 'beta_2': 0.9678634708540109}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:12:39,723] Trial 508 finished with value: 0.668749988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004763365270861491, 'epsilon': 3.183685529494234e-05, 'beta_1': 0.9544687373520163, 'beta_2': 0.9723737519287202}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:12:40,388] Trial 510 finished with value: 0.5562499761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005876638180464655, 'epsilon': 3.2056474561139734e-07, 'beta_1': 0.9174069308471242, 'beta_2': 0.945747028495157}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:13:18,930] Trial 514 finished with value: 0.596875011920929 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0006238057359319009, 'momentum': 0.003945815943829885, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:13:19,927] Trial 512 finished with value: 0.59375 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0005903181624442541, 'momentum': 0.003655754705519357, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:13:20,615] Trial 513 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006139061545087043, 'epsilon': 5.597935325580142e-05, 'beta_1': 0.9540407543838682, 'beta_2': 0.9727958481435859}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:13:20,657] Trial 511 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000590536491501608, 'epsilon': 3.0081321879979215e-07, 'beta_1': 0.9552818460682894, 'beta_2': 0.9718733264945101}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:14:02,559] Trial 517 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00046182527135270047, 'epsilon': 3.3415596643156076e-05, 'beta_1': 0.9573804469978984, 'beta_2': 0.9758028045738899}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:14:03,949] Trial 516 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00044035729277856186, 'epsilon': 3.0178144951180946e-05, 'beta_1': 0.9544353537309629, 'beta_2': 0.972821495515288}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:14:04,236] Trial 515 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00044179012237149196, 'epsilon': 7.489403044635441e-05, 'beta_1': 0.9547157359431657, 'beta_2': 0.9722116934680894}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:14:04,302] Trial 518 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004505204652606412, 'epsilon': 2.382466122633737e-06, 'beta_1': 0.9579186348682089, 'beta_2': 0.9760812911635048}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:14:46,248] Trial 519 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 7.728161438949018e-05, 'epsilon': 2.7365409748565355e-05, 'beta_1': 0.942467899071352, 'beta_2': 0.9763347462037193}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:14:48,360] Trial 520 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035401728942174897, 'epsilon': 3.3069035562869664e-05, 'beta_1': 0.9632409323058584, 'beta_2': 0.9747631626472056}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:14:48,402] Trial 521 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00032878893103146595, 'epsilon': 2.256812151300318e-05, 'beta_1': 0.9520927838097352, 'beta_2': 0.9803798236052178}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:14:48,412] Trial 522 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00033726120570496234, 'epsilon': 2.2719025877773188e-05, 'beta_1': 0.9601158534694365, 'beta_2': 0.9786352425542295}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:15:27,771] Trial 523 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003708586989470952, 'epsilon': 2.339000795434194e-05, 'beta_1': 0.9612631171771925, 'beta_2': 0.9663832448760629}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:15:30,969] Trial 525 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007815218303556548, 'epsilon': 1.4625189405450908e-05, 'beta_1': 0.9616080878319726, 'beta_2': 0.9672454024381233}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:15:31,342] Trial 526 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008976122056949803, 'epsilon': 1.4205708827301922e-05, 'beta_1': 0.9618193688437074, 'beta_2': 0.9695885809181707}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:15:31,628] Trial 524 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003366219748900861, 'epsilon': 3.855699288963429e-07, 'beta_1': 0.9515481669764909, 'beta_2': 0.9695078799814844}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:16:01,832] Trial 527 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00035993076982323777, 'epsilon': 4.141363077583411e-05, 'beta_1': 0.9639189030808083, 'beta_2': 0.9622278095030641}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:16:05,891] Trial 528 finished with value: 0.659375011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008559424033585977, 'epsilon': 1.487760588794869e-06, 'beta_1': 0.9490315887007328, 'beta_2': 0.970516365772921}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:16:05,914] Trial 529 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0013543602262039984, 'epsilon': 4.625069660931451e-07, 'beta_1': 0.9492033623618692, 'beta_2': 0.9187448859771098}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:16:05,960] Trial 530 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00028448951473189585, 'epsilon': 0.00014601571146911435, 'beta_1': 0.9651012202820114, 'beta_2': 0.9875861271080966}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:16:29,141] Trial 531 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005016978521914851, 'epsilon': 9.674527314947426e-07, 'beta_1': 0.9439722401775794, 'beta_2': 0.9413272482741686}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:16:39,475] Trial 533 finished with value: 0.621874988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000507729046211471, 'epsilon': 5.662542427304073e-05, 'beta_1': 0.9574665740357683, 'beta_2': 0.9408734843407142}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:16:39,476] Trial 532 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005065293278252594, 'epsilon': 3.80316817395039e-05, 'beta_1': 0.9581180578873231, 'beta_2': 0.9396849050691902}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:16:39,501] Trial 534 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0011685828724347942, 'epsilon': 1.792139191222878e-05, 'beta_1': 0.9536205172429517, 'beta_2': 0.9434370084151099}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:16:53,927] Trial 535 finished with value: 0.637499988079071 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.001816111754887344, 'momentum': 0.009057806878347542, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:17:12,886] Trial 538 finished with value: 0.612500011920929 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.00029102810458795226, 'momentum': 0.00301205137582727, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:17:12,893] Trial 537 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00038592090528271776, 'epsilon': 2.52495065205996e-05, 'beta_1': 0.9672484591066264, 'beta_2': 0.9610688111582796}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:17:12,893] Trial 536 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0006849233080773598, 'momentum': 0.011576875809649839, 'nesterov': True}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:17:20,254] Trial 539 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003954688400361847, 'epsilon': 3.1227435165863926e-05, 'beta_1': 0.9755156402184074, 'beta_2': 0.9824263519248294}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:17:46,864] Trial 540 finished with value: 0.6000000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009767300823392347, 'epsilon': 7.058381643238026e-07, 'beta_1': 0.9842882690205749, 'beta_2': 0.9501649340518648}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:17:46,866] Trial 542 finished with value: 0.421875 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.09682483384644736, 'epsilon': 7.568118068097396e-07, 'beta_1': 0.9844274266316364, 'beta_2': 0.949379447951986}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:17:46,893] Trial 541 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000737204883724615, 'epsilon': 7.607318570764326e-07, 'beta_1': 0.9452968696670382, 'beta_2': 0.951411258030273}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:17:49,525] Trial 543 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006676181109661854, 'epsilon': 3.438659650282856e-05, 'beta_1': 0.9454477035816621, 'beta_2': 0.9743409339771933}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:18:20,206] Trial 544 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0007618733943665266, 'epsilon': 5.6669018775014755e-05, 'beta_1': 0.9699061081528548, 'beta_2': 0.9725995986964548}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:18:21,918] Trial 545 finished with value: 0.4156250059604645 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0007168289022595167, 'rho': 0.9652530979826708}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:18:22,153] Trial 547 finished with value: 0.2718749940395355 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0005449892717322605, 'rho': 0.9660755469842398}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:18:22,210] Trial 546 finished with value: 0.3843750059604645 and parameters: {'optimizer': 'Adadelta', 'learning_rate': 0.0002878125252259005, 'rho': 0.9668552329851665}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:18:53,013] Trial 548 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000564504329615827, 'epsilon': 2.237160823480192e-06, 'beta_1': 0.9415494176587651, 'beta_2': 0.9452277198317304}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:18:55,351] Trial 549 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005408440696268938, 'epsilon': 4.74342922391403e-07, 'beta_1': 0.9520834708089285, 'beta_2': 0.9644530428396785}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:18:55,384] Trial 551 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0015144987289747336, 'epsilon': 1.5761979513028027e-05, 'beta_1': 0.9532840812772739, 'beta_2': 0.979701724712369}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:18:55,390] Trial 550 finished with value: 0.6468750238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004012032121807901, 'epsilon': 1.9420325969638028e-05, 'beta_1': 0.9561491834249645, 'beta_2': 0.9850982430451929}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:19:23,647] Trial 552 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004071570757982783, 'epsilon': 4.878045763072917e-07, 'beta_1': 0.9629613528661487, 'beta_2': 0.8550778115116167}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:19:28,747] Trial 555 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003380375500076824, 'epsilon': 3.0497947810219064e-07, 'beta_1': 0.9711869854648614, 'beta_2': 0.9750668664077843}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:19:28,750] Trial 553 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00042795749643381427, 'epsilon': 2.4285485582793073e-07, 'beta_1': 0.9602750777644293, 'beta_2': 0.9741303635398614}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:19:28,756] Trial 554 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003686021638634356, 'epsilon': 2.578279690929709e-07, 'beta_1': 0.9727899234874372, 'beta_2': 0.9758462773525665}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:19:49,696] Trial 556 finished with value: 0.6781250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00033686306250726614, 'epsilon': 1.109153011757904e-06, 'beta_1': 0.9475858366269865, 'beta_2': 0.9679366560498618}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:20:02,975] Trial 559 finished with value: 0.612500011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00026542528496523523, 'epsilon': 3.139934150918783e-06, 'beta_1': 0.9484119239720024, 'beta_2': 0.9807930651777153}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:20:03,009] Trial 558 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0002699280906231366, 'epsilon': 1.3047547691807351e-06, 'beta_1': 0.9565199793486696, 'beta_2': 0.9712061363814212}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:20:03,021] Trial 557 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00030605529296369135, 'epsilon': 3.730599603692389e-06, 'beta_1': 0.9566843378206943, 'beta_2': 0.9669193733433162}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:20:15,004] Trial 560 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0009465690169184835, 'epsilon': 1.8694788472557556e-07, 'beta_1': 0.9560464345932671, 'beta_2': 0.9671016324965735}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:20:38,293] Trial 561 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00033687337197915084, 'epsilon': 4.386367186637064e-05, 'beta_1': 0.952135869502402, 'beta_2': 0.9358070744958785}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:20:38,300] Trial 563 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0003279083443561369, 'epsilon': 3.086945117544068e-05, 'beta_1': 0.9549514915415988, 'beta_2': 0.9355408113106255}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:20:38,334] Trial 562 finished with value: 0.543749988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 1.05954382590767e-05, 'epsilon': 2.641647087569342e-05, 'beta_1': 0.9514855995717858, 'beta_2': 0.8607929590304033}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:20:43,753] Trial 564 finished with value: 0.578125 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 1.0384925229026394e-05, 'rho': 0.9198034426891305, 'momentum': 0.01896509730740256, 'epsilon': 1.0108859212291633e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:21:15,017] Trial 566 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00046118253621040787, 'rho': 0.9601819302887691, 'momentum': 0.0025295761976351742, 'epsilon': 1.6725846767178087e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:21:15,018] Trial 565 finished with value: 0.659375011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00046889100435170073, 'rho': 0.9789661278660611, 'momentum': 0.0069955856230125135, 'epsilon': 1.0685294084888043e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:21:15,019] Trial 567 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0004738886050961236, 'epsilon': 1.692777232029402e-06, 'beta_1': 0.9377464659386675, 'beta_2': 0.9603282094540422}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:21:16,529] Trial 568 finished with value: 0.659375011920929 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.00043819962538202564, 'rho': 0.9828671738539996, 'momentum': 4.0017163630476044e-05, 'epsilon': 1.6724969367928285e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:21:49,913] Trial 569 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0005960586265345116, 'epsilon': 8.947793501094859e-05, 'beta_1': 0.9642199709389517, 'beta_2': 0.969817635521784}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:21:50,929] Trial 571 finished with value: 0.643750011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006089629405057119, 'epsilon': 4.1511840351365354e-07, 'beta_1': 0.9641539487913156, 'beta_2': 0.9644904770839526}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:21:51,004] Trial 570 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006116136131319613, 'epsilon': 3.377429550129742e-07, 'beta_1': 0.9630019354233909, 'beta_2': 0.97026154279591}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:21:51,027] Trial 572 finished with value: 0.6625000238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00023384278379819735, 'epsilon': 7.373843972089555e-05, 'beta_1': 0.942254166300873, 'beta_2': 0.963238061338861}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:22:23,094] Trial 573 finished with value: 0.637499988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008335916880169446, 'epsilon': 6.605021161051639e-05, 'beta_1': 0.9635565095969078, 'beta_2': 0.9650966243229908}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:22:25,052] Trial 575 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0010238923610854278, 'momentum': 0.00597663046178845, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:22:25,085] Trial 574 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0011510444096254201, 'epsilon': 5.661362597122199e-07, 'beta_1': 0.9598926445957535, 'beta_2': 0.8567804193884531}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:22:25,112] Trial 576 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0010930803746152398, 'epsilon': 6.448745145331435e-07, 'beta_1': 0.9468870374135476, 'beta_2': 0.945864127733682}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:22:53,495] Trial 577 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'SGD', 'learning_rate': 0.0003997982825503175, 'momentum': 0.005758138929919329, 'nesterov': False}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:22:57,924] Trial 579 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0006870096680454693, 'epsilon': 9.743783200973242e-07, 'beta_1': 0.9475593718115807, 'beta_2': 0.9443712288564111}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:22:57,995] Trial 580 finished with value: 0.6781250238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0016647678188130605, 'epsilon': 9.50465087238257e-07, 'beta_1': 0.9481547145285262, 'beta_2': 0.9455591882882088}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:22:58,022] Trial 578 finished with value: 0.6656249761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.000668542176597082, 'epsilon': 2.3535428863818873e-06, 'beta_1': 0.9479160416151132, 'beta_2': 0.9448324374492895}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:23:20,691] Trial 581 finished with value: 0.628125011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00027594140176440916, 'epsilon': 1.3031662652118152e-06, 'beta_1': 0.9356550649477631, 'beta_2': 0.9578809187419073}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:23:31,045] Trial 582 finished with value: 0.6156250238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0015583688646428797, 'rho': 0.9516944065713172, 'momentum': 0.009497713837912713, 'epsilon': 9.660032729430276e-07}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:23:31,097] Trial 583 finished with value: 0.621874988079071 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.002824760758996984, 'rho': 0.92685137097294, 'momentum': 0.010417645009540129, 'epsilon': 1.4434632970136118e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:23:31,098] Trial 584 finished with value: 0.6000000238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0014515460127522814, 'rho': 0.9266229766151111, 'momentum': 0.008557949179443945, 'epsilon': 1.1949149808274217e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:23:45,789] Trial 585 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'RMSprop', 'learning_rate': 0.0020098989475149957, 'rho': 0.9515663044507687, 'momentum': 0.007867128334088966, 'epsilon': 1.2247392030313975e-06}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:24:04,300] Trial 588 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0018234169354736521, 'epsilon': 6.884401398378043e-07, 'beta_1': 0.950548520763015, 'beta_2': 0.9899439467810001}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:24:04,330] Trial 586 finished with value: 0.578125 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0016653400865733819, 'epsilon': 6.72674606653391e-07, 'beta_1': 0.9511759436862274, 'beta_2': 0.9314321381329251}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:24:04,355] Trial 587 finished with value: 0.625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0014120222172827213, 'epsilon': 6.438607879243745e-07, 'beta_1': 0.9501577905698804, 'beta_2': 0.8622908518848976}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:24:11,671] Trial 589 finished with value: 0.6187499761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0024093375862831324, 'epsilon': 3.8875335784427404e-07, 'beta_1': 0.9434773245236785, 'beta_2': 0.9763512609380699}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:24:36,991] Trial 590 finished with value: 0.653124988079071 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0016175439212645628, 'epsilon': 4.7517766426948456e-05, 'beta_1': 0.9695220458962932, 'beta_2': 0.9623705136675513}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:24:38,046] Trial 592 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0012172958258605778, 'epsilon': 8.170048189452651e-07, 'beta_1': 0.9701721950405353, 'beta_2': 0.9611445499764428}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:24:38,262] Trial 591 finished with value: 0.6312500238418579 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0023215925729231546, 'epsilon': 0.0001634776880904834, 'beta_1': 0.968532841047903, 'beta_2': 0.9377650803612854}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:24:39,975] Trial 593 finished with value: 0.640625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0015214095693798228, 'epsilon': 0.0003984380506927712, 'beta_1': 0.9542010335227582, 'beta_2': 0.9225498565497897}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:25:09,322] Trial 594 finished with value: 0.565625011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0011373768233258162, 'epsilon': 3.0832763128726343e-06, 'beta_1': 0.9447007344561268, 'beta_2': 0.9468713117025056}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:25:11,495] Trial 595 finished with value: 0.6499999761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0012423731253246204, 'epsilon': 2.3837441419960977e-06, 'beta_1': 0.9472298202807737, 'beta_2': 0.9379243857500272}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:25:11,527] Trial 597 finished with value: 0.65625 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0019848183133089436, 'epsilon': 2.4747089756767114e-06, 'beta_1': 0.9583624982053223, 'beta_2': 0.9388589831100591}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:25:11,595] Trial 596 finished with value: 0.6343749761581421 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.00203515973729852, 'epsilon': 2.5714383218139098e-06, 'beta_1': 0.9404005489304702, 'beta_2': 0.9423291391220888}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:25:30,303] Trial 598 finished with value: 0.609375 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0012489503361471752, 'epsilon': 2.4889892411837473e-06, 'beta_1': 0.9465377966571233, 'beta_2': 0.947355324900374}. Best is trial 196 with value: 0.690625011920929. [I 2021-03-10 21:25:31,072] Trial 599 finished with value: 0.612500011920929 and parameters: {'optimizer': 'Adam', 'learning_rate': 0.0008024832404655479, 'epsilon': 8.270500129866154e-07, 'beta_1': 0.9533724129246819, 'beta_2': 0.9432470273517332}. Best is trial 196 with value: 0.690625011920929.
fine_best_params,opt_conf = study_fine.best_params,study_fine.best_params
fine_best_params, study_fine.best_value
({'optimizer': 'Adam',
'learning_rate': 0.000609593339702459,
'epsilon': 4.226899030767284e-06,
'beta_1': 0.9537281643103777,
'beta_2': 0.9390376626945334},
0.690625011920929)
hp_score_board = hp_score_board.append({'Phase':'Fine Tuning - Optimizer',
'Best Value':study_fine.best_value,
'Hyperparameters':f'{study_fine.best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.659375 | {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000} |
| 1 | Coarse Tuning | 0.681250 | {'drop_l1': 0.11769241744475195, 'drop_l2': 0.1203918426967813, 'drop_l3': 0.3079277845699331, 'drop_l4': 0.16249139281479208, 'drop_l5': 0.3503972703488872, 'drop_l6': 0.5575415438784195, 'activation': 'relu', 'kernel_init': 'lecun_uniform'} |
| 2 | Fine Tuning - Optimizer | 0.690625 | {'optimizer': 'Adam', 'learning_rate': 0.000609593339702459, 'epsilon': 4.226899030767284e-06, 'beta_1': 0.9537281643103777, 'beta_2': 0.9390376626945334} |
1% with Adam optimizer and learning rate of 0.0006.## building the optimizer from the fine tuning.
sel_opti = fine_best_params['optimizer']
fine_best_params.pop('optimizer', None)
fine_best_params
optimizer = getattr(tf.optimizers, sel_opti)(**fine_best_params)
optimizer
<tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7ff289ca40a0>
hp_best_params['opts'] = optimizer
hp_best_params
{'num_layers': 6,
'arch': 'A D B',
'neurons_per_layer': [64, 256, 1000, 10, 1000, 1000],
'activation': 'relu',
'kernel_init': 'lecun_uniform',
'drop_rate_per_layer': [0.11769241744475195,
0.1203918426967813,
0.3079277845699331,
0.16249139281479208,
0.3503972703488872,
0.5575415438784195],
'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7ff289ca40a0>}
study_fine.trials_dataframe()
| number | value | datetime_start | datetime_complete | duration | params_beta_1 | params_beta_2 | params_epsilon | params_learning_rate | params_momentum | params_nesterov | params_optimizer | params_rho | state | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.587500 | 2021-03-10 19:46:18.778487 | 2021-03-10 19:46:59.671651 | 0 days 00:00:40.893164 | NaN | NaN | NaN | 0.078210 | 0.000026 | True | SGD | NaN | COMPLETE |
| 1 | 1 | 0.615625 | 2021-03-10 19:46:18.780670 | 2021-03-10 19:46:56.864679 | 0 days 00:00:38.084009 | NaN | NaN | NaN | 0.000574 | 0.088318 | True | SGD | NaN | COMPLETE |
| 2 | 2 | 0.453125 | 2021-03-10 19:46:18.783171 | 2021-03-10 19:47:00.525975 | 0 days 00:00:41.742804 | NaN | NaN | 4.427009e-05 | 0.034562 | 0.000096 | NaN | RMSprop | 0.917819 | COMPLETE |
| 3 | 3 | 0.578125 | 2021-03-10 19:46:18.784546 | 2021-03-10 19:47:00.042512 | 0 days 00:00:41.257966 | NaN | NaN | 4.189703e-09 | 0.000021 | 0.023156 | NaN | RMSprop | 0.891162 | COMPLETE |
| 4 | 4 | 0.600000 | 2021-03-10 19:46:56.871593 | 2021-03-10 19:47:34.110029 | 0 days 00:00:37.238436 | 0.851630 | 0.872424 | 2.474036e-04 | 0.007482 | NaN | NaN | Adam | NaN | COMPLETE |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 595 | 595 | 0.650000 | 2021-03-10 21:24:38.051508 | 2021-03-10 21:25:11.494983 | 0 days 00:00:33.443475 | 0.947230 | 0.937924 | 2.383744e-06 | 0.001242 | NaN | NaN | Adam | NaN | COMPLETE |
| 596 | 596 | 0.634375 | 2021-03-10 21:24:38.272628 | 2021-03-10 21:25:11.595322 | 0 days 00:00:33.322694 | 0.940401 | 0.942329 | 2.571438e-06 | 0.002035 | NaN | NaN | Adam | NaN | COMPLETE |
| 597 | 597 | 0.656250 | 2021-03-10 21:24:39.978561 | 2021-03-10 21:25:11.526967 | 0 days 00:00:31.548406 | 0.958362 | 0.938859 | 2.474709e-06 | 0.001985 | NaN | NaN | Adam | NaN | COMPLETE |
| 598 | 598 | 0.609375 | 2021-03-10 21:25:09.327066 | 2021-03-10 21:25:30.302908 | 0 days 00:00:20.975842 | 0.946538 | 0.947355 | 2.488989e-06 | 0.001249 | NaN | NaN | Adam | NaN | COMPLETE |
| 599 | 599 | 0.612500 | 2021-03-10 21:25:11.498831 | 2021-03-10 21:25:31.072047 | 0 days 00:00:19.573216 | 0.953372 | 0.943247 | 8.270500e-07 | 0.000802 | NaN | NaN | Adam | NaN | COMPLETE |
600 rows × 14 columns
optuna.visualization.plot_parallel_coordinate(study_fine)
optuna.visualization.plot_optimization_history(study_fine)
optuna.visualization.plot_slice(study_fine)
hp_best_params
{'num_layers': 6,
'arch': 'A D B',
'neurons_per_layer': [64, 256, 1000, 10, 1000, 1000],
'activation': 'relu',
'kernel_init': 'lecun_uniform',
'drop_rate_per_layer': [0.11769241744475195,
0.1203918426967813,
0.3079277845699331,
0.16249139281479208,
0.3503972703488872,
0.5575415438784195],
'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7ff289ca40a0>}
def objective_final(trial):
global hp_best_params,loss,metric,X_train,X_test,y_train,y_test
# momentum = trial.suggest_uniform('momentum',0.9,0.99)
# epsilon = trial.suggest_loguniform('epsilon',0.0005,0.05)
# beta_init_std = trial.suggest_categorical('beta_init_std',[1.0,0.5,0.05,0.005,0.005])
# gamma_init = trial.suggest_categorical('gamma_init',[0.8,0.9,1.0])
# center = trial.suggest_categorical('center',[True,False])
# scale = trial.suggest_categorical('scale',[True,False])
batch_size = trial.suggest_int('batch_size',1,100)
n_epochs = trial.suggest_categorical('n_epochs',[50,100,200,500,1000,2000])
kwargs = {
# 'momentum': momentum,
# 'epsilon' :epsilon,
# 'beta_init_std':beta_init_std,
# 'gamma_init': gamma_init,
# 'center': center,
# 'scale': scale,
'batch_size' : batch_size,
'n_epochs' : n_epochs
}
kwargs = {**hp_best_params, **kwargs}
# print(kwargs)
dnn_1 = DNN_Model('classification')
m_1 = dnn_1.create_model(X_train.shape[1],n_class,**kwargs)
# callbacks = LivePlot(2,train_loss='mean_squared_error',train_metric='mean_squared_error')
callbacks=[]
dnn_1.train_model(m_1,X_train,y_train,['sparse_categorical_crossentropy'],['accuracy'],callbacks=[callbacks])
result = dnn_1.test_model(X_test,y_test)
return result[1]
study_final = optuna.create_study(direction='maximize',sampler=optuna.samplers.TPESampler())
study_final.optimize(objective_final,n_trials=500,n_jobs=-13)
[I 2021-03-11 03:12:56,171] A new study created in memory with name: no-name-785b6bad-fa03-4bef-8346-4769449e7ace [I 2021-03-11 03:14:47,863] Trial 1 finished with value: 0.6468750238418579 and parameters: {'batch_size': 53, 'n_epochs': 100}. Best is trial 1 with value: 0.6468750238418579. [I 2021-03-11 03:15:47,218] Trial 194 finished with value: 0.653124988079071 and parameters: {'batch_size': 83, 'n_epochs': 2000}. Best is trial 5 with value: 0.590624988079071. [I 2021-03-11 03:15:47,224] Trial 195 finished with value: 0.640625 and parameters: {'batch_size': 83, 'n_epochs': 2000}. Best is trial 5 with value: 0.590624988079071. [I 2021-03-11 03:16:05,335] Trial 2 finished with value: 0.6468750238418579 and parameters: {'batch_size': 82, 'n_epochs': 200}. Best is trial 1 with value: 0.6468750238418579. [I 2021-03-11 03:16:34,944] Trial 196 finished with value: 0.6312500238418579 and parameters: {'batch_size': 91, 'n_epochs': 2000}. Best is trial 5 with value: 0.590624988079071. [I 2021-03-11 03:16:43,453] Trial 5 finished with value: 0.643750011920929 and parameters: {'batch_size': 42, 'n_epochs': 50}. Best is trial 1 with value: 0.6468750238418579. [I 2021-03-11 03:18:40,137] Trial 3 finished with value: 0.6875 and parameters: {'batch_size': 88, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:19:10,237] Trial 7 finished with value: 0.625 and parameters: {'batch_size': 14, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:20:07,247] Trial 8 finished with value: 0.6312500238418579 and parameters: {'batch_size': 89, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:21:00,788] Trial 6 finished with value: 0.628125011920929 and parameters: {'batch_size': 42, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:22:51,988] Trial 0 finished with value: 0.6312500238418579 and parameters: {'batch_size': 67, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:23:22,989] Trial 11 finished with value: 0.625 and parameters: {'batch_size': 88, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:27:05,220] Trial 197 finished with value: 0.65625 and parameters: {'batch_size': 89, 'n_epochs': 2000}. Best is trial 5 with value: 0.590624988079071. [I 2021-03-11 03:28:00,485] Trial 12 finished with value: 0.6468750238418579 and parameters: {'batch_size': 17, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:28:56,754] Trial 9 finished with value: 0.6499999761581421 and parameters: {'batch_size': 4, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:29:48,545] Trial 10 finished with value: 0.643750011920929 and parameters: {'batch_size': 8, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:32:31,025] Trial 4 finished with value: 0.659375011920929 and parameters: {'batch_size': 76, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:43:49,997] Trial 13 finished with value: 0.643750011920929 and parameters: {'batch_size': 100, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:44:39,539] Trial 14 finished with value: 0.643750011920929 and parameters: {'batch_size': 100, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:45:23,241] Trial 15 finished with value: 0.6468750238418579 and parameters: {'batch_size': 99, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:47:44,346] Trial 16 finished with value: 0.668749988079071 and parameters: {'batch_size': 100, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:49:06,674] Trial 19 finished with value: 0.637499988079071 and parameters: {'batch_size': 70, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:50:35,157] Trial 21 finished with value: 0.637499988079071 and parameters: {'batch_size': 54, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:51:24,048] Trial 20 finished with value: 0.6468750238418579 and parameters: {'batch_size': 63, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:54:15,721] Trial 22 finished with value: 0.6656249761581421 and parameters: {'batch_size': 92, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:55:08,417] Trial 23 finished with value: 0.6000000238418579 and parameters: {'batch_size': 91, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:58:00,275] Trial 24 finished with value: 0.671875 and parameters: {'batch_size': 91, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:58:23,166] Trial 17 finished with value: 0.637499988079071 and parameters: {'batch_size': 70, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:58:50,191] Trial 25 finished with value: 0.659375011920929 and parameters: {'batch_size': 80, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 03:59:12,610] Trial 18 finished with value: 0.6656249761581421 and parameters: {'batch_size': 71, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:01:44,036] Trial 26 finished with value: 0.643750011920929 and parameters: {'batch_size': 79, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:02:07,304] Trial 27 finished with value: 0.640625 and parameters: {'batch_size': 81, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:02:34,191] Trial 28 finished with value: 0.628125011920929 and parameters: {'batch_size': 99, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:02:57,078] Trial 31 finished with value: 0.6343749761581421 and parameters: {'batch_size': 97, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:02:57,544] Trial 29 finished with value: 0.6625000238418579 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:03:21,670] Trial 32 finished with value: 0.653124988079071 and parameters: {'batch_size': 59, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:04:28,890] Trial 33 finished with value: 0.659375011920929 and parameters: {'batch_size': 61, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:05:22,385] Trial 30 finished with value: 0.6656249761581421 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:06:57,213] Trial 35 finished with value: 0.621874988079071 and parameters: {'batch_size': 91, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:08:03,897] Trial 36 finished with value: 0.659375011920929 and parameters: {'batch_size': 91, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:09:00,873] Trial 37 finished with value: 0.6312500238418579 and parameters: {'batch_size': 85, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:14:07,569] Trial 38 finished with value: 0.6625000238418579 and parameters: {'batch_size': 87, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:16:12,562] Trial 40 finished with value: 0.6343749761581421 and parameters: {'batch_size': 94, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:16:40,752] Trial 42 finished with value: 0.625 and parameters: {'batch_size': 76, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:17:16,865] Trial 34 finished with value: 0.65625 and parameters: {'batch_size': 65, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:22:22,083] Trial 39 finished with value: 0.609375 and parameters: {'batch_size': 73, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:28:53,381] Trial 41 finished with value: 0.625 and parameters: {'batch_size': 74, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:31:40,913] Trial 43 finished with value: 0.659375011920929 and parameters: {'batch_size': 42, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:32:16,420] Trial 44 finished with value: 0.628125011920929 and parameters: {'batch_size': 72, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:37:53,470] Trial 45 finished with value: 0.668749988079071 and parameters: {'batch_size': 46, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:38:22,531] Trial 49 finished with value: 0.6468750238418579 and parameters: {'batch_size': 30, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:46:27,861] Trial 46 finished with value: 0.625 and parameters: {'batch_size': 44, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:50:54,634] Trial 47 finished with value: 0.628125011920929 and parameters: {'batch_size': 84, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:51:57,776] Trial 48 finished with value: 0.606249988079071 and parameters: {'batch_size': 84, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:53:31,001] Trial 52 finished with value: 0.637499988079071 and parameters: {'batch_size': 37, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:54:29,348] Trial 53 finished with value: 0.6499999761581421 and parameters: {'batch_size': 50, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 04:59:39,665] Trial 54 finished with value: 0.6468750238418579 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:00:36,951] Trial 55 finished with value: 0.6656249761581421 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:01:21,713] Trial 50 finished with value: 0.6343749761581421 and parameters: {'batch_size': 38, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:06:49,517] Trial 57 finished with value: 0.671875 and parameters: {'batch_size': 26, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:07:31,706] Trial 58 finished with value: 0.625 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:10:53,793] Trial 51 finished with value: 0.643750011920929 and parameters: {'batch_size': 50, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:13:09,941] Trial 59 finished with value: 0.675000011920929 and parameters: {'batch_size': 15, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:13:57,422] Trial 60 finished with value: 0.671875 and parameters: {'batch_size': 20, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:17:13,362] Trial 61 finished with value: 0.6156250238418579 and parameters: {'batch_size': 21, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:19:37,268] Trial 62 finished with value: 0.6343749761581421 and parameters: {'batch_size': 16, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:20:24,619] Trial 63 finished with value: 0.59375 and parameters: {'batch_size': 19, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:23:31,552] Trial 64 finished with value: 0.609375 and parameters: {'batch_size': 24, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:24:31,124] Trial 56 finished with value: 0.640625 and parameters: {'batch_size': 29, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:24:55,326] Trial 67 finished with value: 0.6031249761581421 and parameters: {'batch_size': 1, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:25:51,489] Trial 65 finished with value: 0.653124988079071 and parameters: {'batch_size': 10, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:25:54,565] Trial 68 finished with value: 0.640625 and parameters: {'batch_size': 11, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:26:37,544] Trial 66 finished with value: 0.6625000238418579 and parameters: {'batch_size': 8, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:31:10,823] Trial 69 finished with value: 0.653124988079071 and parameters: {'batch_size': 12, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:32:10,596] Trial 70 finished with value: 0.6499999761581421 and parameters: {'batch_size': 11, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:32:16,206] Trial 71 finished with value: 0.6499999761581421 and parameters: {'batch_size': 6, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:32:52,937] Trial 72 finished with value: 0.6468750238418579 and parameters: {'batch_size': 25, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:37:29,876] Trial 73 finished with value: 0.653124988079071 and parameters: {'batch_size': 24, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:38:25,851] Trial 74 finished with value: 0.621874988079071 and parameters: {'batch_size': 25, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:38:32,562] Trial 75 finished with value: 0.675000011920929 and parameters: {'batch_size': 26, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:38:58,170] Trial 76 finished with value: 0.637499988079071 and parameters: {'batch_size': 32, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:43:34,158] Trial 77 finished with value: 0.653124988079071 and parameters: {'batch_size': 33, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:50:23,656] Trial 78 finished with value: 0.6343749761581421 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:50:29,644] Trial 79 finished with value: 0.6656249761581421 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:50:45,986] Trial 80 finished with value: 0.643750011920929 and parameters: {'batch_size': 34, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:51:05,172] Trial 82 finished with value: 0.6499999761581421 and parameters: {'batch_size': 15, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:51:12,105] Trial 83 finished with value: 0.609375 and parameters: {'batch_size': 22, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:55:28,317] Trial 81 finished with value: 0.684374988079071 and parameters: {'batch_size': 17, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:56:58,085] Trial 84 finished with value: 0.640625 and parameters: {'batch_size': 20, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 05:57:17,802] Trial 85 finished with value: 0.6156250238418579 and parameters: {'batch_size': 19, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:07:52,844] Trial 87 finished with value: 0.6656249761581421 and parameters: {'batch_size': 17, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:09:26,395] Trial 88 finished with value: 0.671875 and parameters: {'batch_size': 17, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:09:48,243] Trial 89 finished with value: 0.65625 and parameters: {'batch_size': 47, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:16:04,438] Trial 86 finished with value: 0.6625000238418579 and parameters: {'batch_size': 58, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:20:41,002] Trial 90 finished with value: 0.640625 and parameters: {'batch_size': 56, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:22:15,810] Trial 91 finished with value: 0.671875 and parameters: {'batch_size': 27, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:22:31,084] Trial 92 finished with value: 0.637499988079071 and parameters: {'batch_size': 57, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:29:15,654] Trial 93 finished with value: 0.659375011920929 and parameters: {'batch_size': 27, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:33:54,346] Trial 94 finished with value: 0.6625000238418579 and parameters: {'batch_size': 27, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:35:18,332] Trial 95 finished with value: 0.637499988079071 and parameters: {'batch_size': 14, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:35:31,350] Trial 96 finished with value: 0.6468750238418579 and parameters: {'batch_size': 27, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:42:05,230] Trial 97 finished with value: 0.621874988079071 and parameters: {'batch_size': 18, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:44:38,127] Trial 101 finished with value: 0.65625 and parameters: {'batch_size': 14, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:46:22,900] Trial 98 finished with value: 0.653124988079071 and parameters: {'batch_size': 13, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:47:41,724] Trial 99 finished with value: 0.65625 and parameters: {'batch_size': 18, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:47:50,061] Trial 100 finished with value: 0.6625000238418579 and parameters: {'batch_size': 18, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:50:32,689] Trial 102 finished with value: 0.6343749761581421 and parameters: {'batch_size': 3, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:52:15,530] Trial 103 finished with value: 0.625 and parameters: {'batch_size': 3, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:53:30,638] Trial 104 finished with value: 0.6468750238418579 and parameters: {'batch_size': 21, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 06:53:43,008] Trial 105 finished with value: 0.6499999761581421 and parameters: {'batch_size': 22, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:14:22,491] Trial 106 finished with value: 0.6499999761581421 and parameters: {'batch_size': 68, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:16:20,826] Trial 107 finished with value: 0.65625 and parameters: {'batch_size': 23, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:17:30,603] Trial 108 finished with value: 0.6468750238418579 and parameters: {'batch_size': 68, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:17:30,643] Trial 109 finished with value: 0.6625000238418579 and parameters: {'batch_size': 23, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:17:40,494] Trial 111 finished with value: 0.6156250238418579 and parameters: {'batch_size': 38, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:18:55,304] Trial 112 finished with value: 0.643750011920929 and parameters: {'batch_size': 37, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:23:54,422] Trial 113 finished with value: 0.653124988079071 and parameters: {'batch_size': 89, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:24:06,874] Trial 114 finished with value: 0.6468750238418579 and parameters: {'batch_size': 9, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:30:32,232] Trial 117 finished with value: 0.637499988079071 and parameters: {'batch_size': 98, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:31:28,030] Trial 115 finished with value: 0.6499999761581421 and parameters: {'batch_size': 35, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:36:37,677] Trial 118 finished with value: 0.653124988079071 and parameters: {'batch_size': 92, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:38:56,136] Trial 110 finished with value: 0.6468750238418579 and parameters: {'batch_size': 88, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:43:19,296] Trial 119 finished with value: 0.653124988079071 and parameters: {'batch_size': 16, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:44:43,497] Trial 121 finished with value: 0.637499988079071 and parameters: {'batch_size': 92, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:47:47,111] Trial 116 finished with value: 0.653124988079071 and parameters: {'batch_size': 78, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:48:15,489] Trial 120 finished with value: 0.6625000238418579 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:49:04,632] Trial 122 finished with value: 0.6468750238418579 and parameters: {'batch_size': 78, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:53:35,966] Trial 124 finished with value: 0.668749988079071 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:54:49,805] Trial 126 finished with value: 0.668749988079071 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:56:07,588] Trial 123 finished with value: 0.6468750238418579 and parameters: {'batch_size': 7, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:59:25,627] Trial 127 finished with value: 0.643750011920929 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 07:59:42,693] Trial 125 finished with value: 0.6781250238418579 and parameters: {'batch_size': 16, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:00:39,266] Trial 128 finished with value: 0.6312500238418579 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:02:10,904] Trial 129 finished with value: 0.612500011920929 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:05:47,067] Trial 130 finished with value: 0.625 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:06:06,171] Trial 131 finished with value: 0.6656249761581421 and parameters: {'batch_size': 99, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:07:09,540] Trial 132 finished with value: 0.671875 and parameters: {'batch_size': 20, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:08:48,379] Trial 133 finished with value: 0.637499988079071 and parameters: {'batch_size': 26, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:11:32,126] Trial 137 finished with value: 0.6781250238418579 and parameters: {'batch_size': 16, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:12:22,279] Trial 134 finished with value: 0.628125011920929 and parameters: {'batch_size': 86, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:12:40,543] Trial 135 finished with value: 0.6312500238418579 and parameters: {'batch_size': 86, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:13:36,778] Trial 136 finished with value: 0.659375011920929 and parameters: {'batch_size': 16, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:14:07,299] Trial 138 finished with value: 0.637499988079071 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:15:00,329] Trial 139 finished with value: 0.6781250238418579 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:15:17,023] Trial 140 finished with value: 0.6499999761581421 and parameters: {'batch_size': 12, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:16:10,152] Trial 141 finished with value: 0.628125011920929 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:16:38,482] Trial 142 finished with value: 0.6499999761581421 and parameters: {'batch_size': 13, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:17:29,201] Trial 143 finished with value: 0.6656249761581421 and parameters: {'batch_size': 13, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:17:46,847] Trial 144 finished with value: 0.640625 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:18:40,083] Trial 145 finished with value: 0.653124988079071 and parameters: {'batch_size': 15, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:19:11,079] Trial 146 finished with value: 0.6312500238418579 and parameters: {'batch_size': 15, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:20:00,192] Trial 147 finished with value: 0.609375 and parameters: {'batch_size': 16, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:20:16,988] Trial 148 finished with value: 0.6187499761581421 and parameters: {'batch_size': 24, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:20:41,700] Trial 151 finished with value: 0.640625 and parameters: {'batch_size': 23, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:20:58,518] Trial 152 finished with value: 0.640625 and parameters: {'batch_size': 22, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:21:07,162] Trial 149 finished with value: 0.637499988079071 and parameters: {'batch_size': 18, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:30:42,276] Trial 150 finished with value: 0.625 and parameters: {'batch_size': 17, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:32:15,674] Trial 153 finished with value: 0.65625 and parameters: {'batch_size': 18, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:32:33,446] Trial 154 finished with value: 0.590624988079071 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:32:46,394] Trial 155 finished with value: 0.653124988079071 and parameters: {'batch_size': 10, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:38:27,841] Trial 158 finished with value: 0.606249988079071 and parameters: {'batch_size': 31, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:38:39,967] Trial 159 finished with value: 0.653124988079071 and parameters: {'batch_size': 21, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:42:30,451] Trial 156 finished with value: 0.6187499761581421 and parameters: {'batch_size': 10, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:44:02,330] Trial 157 finished with value: 0.65625 and parameters: {'batch_size': 41, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:44:31,181] Trial 160 finished with value: 0.653124988079071 and parameters: {'batch_size': 25, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:48:58,886] Trial 162 finished with value: 0.659375011920929 and parameters: {'batch_size': 25, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:50:33,978] Trial 163 finished with value: 0.612500011920929 and parameters: {'batch_size': 90, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:51:08,732] Trial 164 finished with value: 0.643750011920929 and parameters: {'batch_size': 90, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:55:50,699] Trial 165 finished with value: 0.6343749761581421 and parameters: {'batch_size': 62, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:57:27,000] Trial 166 finished with value: 0.612500011920929 and parameters: {'batch_size': 94, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 08:57:54,525] Trial 167 finished with value: 0.653124988079071 and parameters: {'batch_size': 93, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:04:09,986] Trial 161 finished with value: 0.6625000238418579 and parameters: {'batch_size': 24, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:09:17,952] Trial 171 finished with value: 0.628125011920929 and parameters: {'batch_size': 54, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:18:50,539] Trial 168 finished with value: 0.637499988079071 and parameters: {'batch_size': 19, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:20:28,102] Trial 169 finished with value: 0.640625 and parameters: {'batch_size': 93, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:20:38,955] Trial 170 finished with value: 0.643750011920929 and parameters: {'batch_size': 19, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:23:05,899] Trial 175 finished with value: 0.637499988079071 and parameters: {'batch_size': 12, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:24:42,502] Trial 173 finished with value: 0.659375011920929 and parameters: {'batch_size': 14, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:26:20,021] Trial 174 finished with value: 0.640625 and parameters: {'batch_size': 99, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:29:01,938] Trial 176 finished with value: 0.606249988079071 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:30:43,329] Trial 177 finished with value: 0.6499999761581421 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:32:19,056] Trial 172 finished with value: 0.637499988079071 and parameters: {'batch_size': 20, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:35:01,742] Trial 179 finished with value: 0.6499999761581421 and parameters: {'batch_size': 17, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:38:12,168] Trial 178 finished with value: 0.637499988079071 and parameters: {'batch_size': 16, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:44:06,157] Trial 183 finished with value: 0.653124988079071 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:46:50,424] Trial 182 finished with value: 0.6625000238418579 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:54:21,887] Trial 180 finished with value: 0.65625 and parameters: {'batch_size': 16, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:55:57,683] Trial 184 finished with value: 0.659375011920929 and parameters: {'batch_size': 83, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:55:59,241] Trial 181 finished with value: 0.65625 and parameters: {'batch_size': 82, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:57:17,466] Trial 188 finished with value: 0.675000011920929 and parameters: {'batch_size': 98, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:58:34,673] Trial 189 finished with value: 0.6625000238418579 and parameters: {'batch_size': 22, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 09:58:39,172] Trial 185 finished with value: 0.653124988079071 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:04:41,550] Trial 190 finished with value: 0.640625 and parameters: {'batch_size': 47, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:04:48,466] Trial 191 finished with value: 0.653124988079071 and parameters: {'batch_size': 94, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:06:07,791] Trial 193 finished with value: 0.653124988079071 and parameters: {'batch_size': 14, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:06:17,883] Trial 186 finished with value: 0.6625000238418579 and parameters: {'batch_size': 82, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:07:26,289] Trial 194 finished with value: 0.596875011920929 and parameters: {'batch_size': 97, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:07:36,913] Trial 195 finished with value: 0.6499999761581421 and parameters: {'batch_size': 98, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:07:55,136] Trial 187 finished with value: 0.6312500238418579 and parameters: {'batch_size': 98, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:08:45,806] Trial 196 finished with value: 0.574999988079071 and parameters: {'batch_size': 100, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:09:13,278] Trial 198 finished with value: 0.640625 and parameters: {'batch_size': 26, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:10:02,160] Trial 197 finished with value: 0.6625000238418579 and parameters: {'batch_size': 12, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:10:33,036] Trial 192 finished with value: 0.609375 and parameters: {'batch_size': 98, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:12:27,119] Trial 201 finished with value: 0.625 and parameters: {'batch_size': 71, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:14:39,588] Trial 199 finished with value: 0.653124988079071 and parameters: {'batch_size': 34, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:15:05,189] Trial 200 finished with value: 0.6468750238418579 and parameters: {'batch_size': 21, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:17:10,845] Trial 204 finished with value: 0.640625 and parameters: {'batch_size': 95, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:19:38,963] Trial 206 finished with value: 0.643750011920929 and parameters: {'batch_size': 75, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:22:27,399] Trial 202 finished with value: 0.640625 and parameters: {'batch_size': 18, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:24:24,423] Trial 203 finished with value: 0.671875 and parameters: {'batch_size': 18, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:25:38,539] Trial 207 finished with value: 0.6625000238418579 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:27:00,508] Trial 205 finished with value: 0.628125011920929 and parameters: {'batch_size': 75, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:28:28,402] Trial 208 finished with value: 0.637499988079071 and parameters: {'batch_size': 90, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:31:38,537] Trial 210 finished with value: 0.6156250238418579 and parameters: {'batch_size': 92, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:32:55,764] Trial 211 finished with value: 0.6656249761581421 and parameters: {'batch_size': 91, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:36:17,875] Trial 209 finished with value: 0.628125011920929 and parameters: {'batch_size': 17, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:38:55,766] Trial 214 finished with value: 0.606249988079071 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:40:20,710] Trial 212 finished with value: 0.637499988079071 and parameters: {'batch_size': 14, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:42:17,604] Trial 215 finished with value: 0.640625 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:43:43,853] Trial 213 finished with value: 0.643750011920929 and parameters: {'batch_size': 17, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:44:54,020] Trial 216 finished with value: 0.6625000238418579 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:46:23,235] Trial 217 finished with value: 0.643750011920929 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:54:13,235] Trial 218 finished with value: 0.6187499761581421 and parameters: {'batch_size': 19, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:55:34,423] Trial 219 finished with value: 0.640625 and parameters: {'batch_size': 22, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 10:58:07,510] Trial 221 finished with value: 0.653124988079071 and parameters: {'batch_size': 20, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:00:26,730] Trial 224 finished with value: 0.659375011920929 and parameters: {'batch_size': 15, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:06:12,690] Trial 225 finished with value: 0.6343749761581421 and parameters: {'batch_size': 13, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:08:11,167] Trial 220 finished with value: 0.643750011920929 and parameters: {'batch_size': 21, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:12:14,555] Trial 226 finished with value: 0.6812499761581421 and parameters: {'batch_size': 92, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:14:04,851] Trial 227 finished with value: 0.640625 and parameters: {'batch_size': 92, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:16:29,868] Trial 229 finished with value: 0.643750011920929 and parameters: {'batch_size': 16, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:17:32,801] Trial 222 finished with value: 0.659375011920929 and parameters: {'batch_size': 23, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:18:09,298] Trial 228 finished with value: 0.6468750238418579 and parameters: {'batch_size': 65, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:18:52,794] Trial 232 finished with value: 0.6499999761581421 and parameters: {'batch_size': 88, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:18:59,889] Trial 223 finished with value: 0.65625 and parameters: {'batch_size': 15, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:22:25,955] Trial 230 finished with value: 0.6343749761581421 and parameters: {'batch_size': 27, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:24:55,854] Trial 233 finished with value: 0.637499988079071 and parameters: {'batch_size': 11, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:29:24,497] Trial 231 finished with value: 0.65625 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:30:53,966] Trial 236 finished with value: 0.628125011920929 and parameters: {'batch_size': 90, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:31:00,979] Trial 234 finished with value: 0.637499988079071 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:34:21,958] Trial 235 finished with value: 0.625 and parameters: {'batch_size': 19, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:35:22,702] Trial 237 finished with value: 0.653124988079071 and parameters: {'batch_size': 17, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:36:51,732] Trial 238 finished with value: 0.6781250238418579 and parameters: {'batch_size': 94, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:37:01,339] Trial 239 finished with value: 0.6343749761581421 and parameters: {'batch_size': 93, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:37:46,677] Trial 241 finished with value: 0.668749988079071 and parameters: {'batch_size': 87, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:40:21,779] Trial 240 finished with value: 0.6812499761581421 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:42:47,033] Trial 242 finished with value: 0.640625 and parameters: {'batch_size': 98, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:42:56,707] Trial 243 finished with value: 0.621874988079071 and parameters: {'batch_size': 88, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:45:25,721] Trial 246 finished with value: 0.6468750238418579 and parameters: {'batch_size': 87, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:46:26,531] Trial 245 finished with value: 0.643750011920929 and parameters: {'batch_size': 88, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:49:44,895] Trial 244 finished with value: 0.6343749761581421 and parameters: {'batch_size': 88, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:51:32,494] Trial 248 finished with value: 0.684374988079071 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:52:28,669] Trial 249 finished with value: 0.6656249761581421 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:55:00,979] Trial 247 finished with value: 0.6312500238418579 and parameters: {'batch_size': 94, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:55:48,120] Trial 250 finished with value: 0.640625 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:57:37,516] Trial 251 finished with value: 0.6187499761581421 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 11:58:29,006] Trial 252 finished with value: 0.6156250238418579 and parameters: {'batch_size': 94, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:01:07,834] Trial 253 finished with value: 0.6468750238418579 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:01:57,832] Trial 254 finished with value: 0.668749988079071 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:02:23,763] Trial 257 finished with value: 0.6468750238418579 and parameters: {'batch_size': 92, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:03:46,257] Trial 255 finished with value: 0.6499999761581421 and parameters: {'batch_size': 91, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:04:35,390] Trial 256 finished with value: 0.643750011920929 and parameters: {'batch_size': 92, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:07:59,931] Trial 258 finished with value: 0.612500011920929 and parameters: {'batch_size': 91, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:08:25,149] Trial 259 finished with value: 0.6499999761581421 and parameters: {'batch_size': 93, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:09:49,128] Trial 260 finished with value: 0.668749988079071 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:10:36,672] Trial 261 finished with value: 0.643750011920929 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:10:52,861] Trial 263 finished with value: 0.6343749761581421 and parameters: {'batch_size': 98, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:13:09,114] Trial 265 finished with value: 0.625 and parameters: {'batch_size': 18, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:14:05,126] Trial 262 finished with value: 0.612500011920929 and parameters: {'batch_size': 98, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:15:55,406] Trial 264 finished with value: 0.6312500238418579 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:16:55,897] Trial 266 finished with value: 0.637499988079071 and parameters: {'batch_size': 94, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:19:17,770] Trial 267 finished with value: 0.643750011920929 and parameters: {'batch_size': 94, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:20:10,691] Trial 268 finished with value: 0.6499999761581421 and parameters: {'batch_size': 20, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:21:52,746] Trial 269 finished with value: 0.6468750238418579 and parameters: {'batch_size': 85, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:22:55,821] Trial 270 finished with value: 0.6625000238418579 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:23:07,266] Trial 273 finished with value: 0.606249988079071 and parameters: {'batch_size': 100, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:23:39,160] Trial 274 finished with value: 0.653124988079071 and parameters: {'batch_size': 24, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:23:49,558] Trial 275 finished with value: 0.643750011920929 and parameters: {'batch_size': 23, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:25:20,993] Trial 271 finished with value: 0.628125011920929 and parameters: {'batch_size': 24, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:26:09,347] Trial 272 finished with value: 0.6343749761581421 and parameters: {'batch_size': 60, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:26:10,159] Trial 276 finished with value: 0.668749988079071 and parameters: {'batch_size': 44, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:26:21,715] Trial 277 finished with value: 0.6468750238418579 and parameters: {'batch_size': 51, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:27:53,119] Trial 278 finished with value: 0.6312500238418579 and parameters: {'batch_size': 18, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:28:43,102] Trial 280 finished with value: 0.640625 and parameters: {'batch_size': 43, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:28:44,081] Trial 279 finished with value: 0.643750011920929 and parameters: {'batch_size': 19, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:30:17,655] Trial 282 finished with value: 0.65625 and parameters: {'batch_size': 46, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:32:21,226] Trial 281 finished with value: 0.6312500238418579 and parameters: {'batch_size': 50, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:32:50,773] Trial 285 finished with value: 0.6187499761581421 and parameters: {'batch_size': 42, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:34:47,086] Trial 284 finished with value: 0.6343749761581421 and parameters: {'batch_size': 15, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:34:48,880] Trial 286 finished with value: 0.653124988079071 and parameters: {'batch_size': 39, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:34:49,846] Trial 283 finished with value: 0.65625 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:56:37,975] Trial 287 finished with value: 0.6343749761581421 and parameters: {'batch_size': 48, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:58:21,774] Trial 289 finished with value: 0.637499988079071 and parameters: {'batch_size': 14, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:58:22,198] Trial 290 finished with value: 0.625 and parameters: {'batch_size': 21, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 12:58:26,307] Trial 288 finished with value: 0.643750011920929 and parameters: {'batch_size': 45, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:04:36,142] Trial 292 finished with value: 0.659375011920929 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:08:48,313] Trial 291 finished with value: 0.625 and parameters: {'batch_size': 21, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:10:36,018] Trial 293 finished with value: 0.659375011920929 and parameters: {'batch_size': 18, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:10:45,059] Trial 294 finished with value: 0.6625000238418579 and parameters: {'batch_size': 16, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:10:53,720] Trial 295 finished with value: 0.628125011920929 and parameters: {'batch_size': 16, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:11:51,355] Trial 297 finished with value: 0.643750011920929 and parameters: {'batch_size': 32, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:12:12,173] Trial 299 finished with value: 0.653124988079071 and parameters: {'batch_size': 32, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:14:21,805] Trial 300 finished with value: 0.653124988079071 and parameters: {'batch_size': 40, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:14:45,385] Trial 301 finished with value: 0.643750011920929 and parameters: {'batch_size': 40, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:14:56,183] Trial 296 finished with value: 0.637499988079071 and parameters: {'batch_size': 17, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:16:48,679] Trial 298 finished with value: 0.6343749761581421 and parameters: {'batch_size': 89, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:20:33,133] Trial 302 finished with value: 0.6468750238418579 and parameters: {'batch_size': 26, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:20:50,409] Trial 303 finished with value: 0.65625 and parameters: {'batch_size': 90, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:21:05,595] Trial 304 finished with value: 0.643750011920929 and parameters: {'batch_size': 90, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:26:40,917] Trial 306 finished with value: 0.640625 and parameters: {'batch_size': 93, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:29:09,219] Trial 305 finished with value: 0.659375011920929 and parameters: {'batch_size': 54, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:33:35,538] Trial 307 finished with value: 0.6343749761581421 and parameters: {'batch_size': 53, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:33:35,554] Trial 308 finished with value: 0.6187499761581421 and parameters: {'batch_size': 13, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:35:36,371] Trial 310 finished with value: 0.637499988079071 and parameters: {'batch_size': 36, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:36:09,604] Trial 311 finished with value: 0.6468750238418579 and parameters: {'batch_size': 20, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:36:11,522] Trial 312 finished with value: 0.637499988079071 and parameters: {'batch_size': 36, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:38:40,831] Trial 314 finished with value: 0.65625 and parameters: {'batch_size': 19, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:39:08,280] Trial 309 finished with value: 0.6499999761581421 and parameters: {'batch_size': 14, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:41:38,592] Trial 313 finished with value: 0.637499988079071 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:42:13,037] Trial 315 finished with value: 0.643750011920929 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:44:41,763] Trial 316 finished with value: 0.637499988079071 and parameters: {'batch_size': 22, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:45:13,673] Trial 317 finished with value: 0.653124988079071 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:45:57,627] Trial 320 finished with value: 0.6343749761581421 and parameters: {'batch_size': 48, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:47:36,514] Trial 318 finished with value: 0.625 and parameters: {'batch_size': 95, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:48:18,683] Trial 319 finished with value: 0.671875 and parameters: {'batch_size': 99, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:51:13,432] Trial 321 finished with value: 0.675000011920929 and parameters: {'batch_size': 44, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:51:58,060] Trial 322 finished with value: 0.675000011920929 and parameters: {'batch_size': 99, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:53:40,483] Trial 323 finished with value: 0.640625 and parameters: {'batch_size': 12, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:54:22,850] Trial 324 finished with value: 0.637499988079071 and parameters: {'batch_size': 97, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:57:21,957] Trial 325 finished with value: 0.621874988079071 and parameters: {'batch_size': 8, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:58:04,007] Trial 326 finished with value: 0.643750011920929 and parameters: {'batch_size': 99, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 13:59:49,059] Trial 327 finished with value: 0.640625 and parameters: {'batch_size': 85, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:00:30,401] Trial 328 finished with value: 0.621874988079071 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:03:27,753] Trial 329 finished with value: 0.6312500238418579 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:04:14,873] Trial 330 finished with value: 0.640625 and parameters: {'batch_size': 98, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:05:51,697] Trial 331 finished with value: 0.6625000238418579 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:09:35,620] Trial 333 finished with value: 0.643750011920929 and parameters: {'batch_size': 26, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:10:27,305] Trial 334 finished with value: 0.6625000238418579 and parameters: {'batch_size': 25, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:12:02,204] Trial 335 finished with value: 0.668749988079071 and parameters: {'batch_size': 17, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:12:33,127] Trial 332 finished with value: 0.659375011920929 and parameters: {'batch_size': 47, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:16:31,845] Trial 337 finished with value: 0.6499999761581421 and parameters: {'batch_size': 16, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:18:33,057] Trial 339 finished with value: 0.653124988079071 and parameters: {'batch_size': 15, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:21:38,237] Trial 336 finished with value: 0.6625000238418579 and parameters: {'batch_size': 48, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:22:36,572] Trial 340 finished with value: 0.6343749761581421 and parameters: {'batch_size': 21, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:23:52,453] Trial 338 finished with value: 0.628125011920929 and parameters: {'batch_size': 22, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:29:49,426] Trial 344 finished with value: 0.606249988079071 and parameters: {'batch_size': 93, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:30:33,859] Trial 341 finished with value: 0.628125011920929 and parameters: {'batch_size': 43, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:34:22,552] Trial 343 finished with value: 0.6625000238418579 and parameters: {'batch_size': 92, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:35:50,217] Trial 345 finished with value: 0.653124988079071 and parameters: {'batch_size': 11, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:36:35,607] Trial 346 finished with value: 0.643750011920929 and parameters: {'batch_size': 18, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:37:08,715] Trial 348 finished with value: 0.65625 and parameters: {'batch_size': 19, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:40:31,800] Trial 347 finished with value: 0.628125011920929 and parameters: {'batch_size': 99, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:41:16,036] Trial 351 finished with value: 0.637499988079071 and parameters: {'batch_size': 94, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:42:48,901] Trial 349 finished with value: 0.6343749761581421 and parameters: {'batch_size': 98, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:43:17,750] Trial 350 finished with value: 0.653124988079071 and parameters: {'batch_size': 87, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:45:53,200] Trial 342 finished with value: 0.6499999761581421 and parameters: {'batch_size': 19, 'n_epochs': 2000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:47:23,736] Trial 352 finished with value: 0.6781250238418579 and parameters: {'batch_size': 17, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:48:58,502] Trial 353 finished with value: 0.640625 and parameters: {'batch_size': 14, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:51:56,332] Trial 355 finished with value: 0.637499988079071 and parameters: {'batch_size': 16, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:53:37,453] Trial 356 finished with value: 0.659375011920929 and parameters: {'batch_size': 14, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:55:20,247] Trial 354 finished with value: 0.6468750238418579 and parameters: {'batch_size': 46, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:58:05,764] Trial 358 finished with value: 0.6468750238418579 and parameters: {'batch_size': 17, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 14:59:46,175] Trial 359 finished with value: 0.675000011920929 and parameters: {'batch_size': 23, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:01:02,189] Trial 357 finished with value: 0.637499988079071 and parameters: {'batch_size': 17, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:01:21,276] Trial 360 finished with value: 0.628125011920929 and parameters: {'batch_size': 17, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:04:07,297] Trial 361 finished with value: 0.6625000238418579 and parameters: {'batch_size': 22, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:05:51,030] Trial 362 finished with value: 0.653124988079071 and parameters: {'batch_size': 23, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:07:09,908] Trial 363 finished with value: 0.6625000238418579 and parameters: {'batch_size': 22, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:07:10,627] Trial 366 finished with value: 0.653124988079071 and parameters: {'batch_size': 24, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:07:25,918] Trial 364 finished with value: 0.621874988079071 and parameters: {'batch_size': 21, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:08:31,544] Trial 367 finished with value: 0.6499999761581421 and parameters: {'batch_size': 25, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:10:16,384] Trial 365 finished with value: 0.6343749761581421 and parameters: {'batch_size': 29, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:13:18,183] Trial 368 finished with value: 0.643750011920929 and parameters: {'batch_size': 20, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:13:28,685] Trial 369 finished with value: 0.628125011920929 and parameters: {'batch_size': 19, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:14:39,612] Trial 370 finished with value: 0.6343749761581421 and parameters: {'batch_size': 27, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:16:21,515] Trial 371 finished with value: 0.609375 and parameters: {'batch_size': 24, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:19:26,905] Trial 372 finished with value: 0.671875 and parameters: {'batch_size': 27, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:19:29,367] Trial 373 finished with value: 0.6343749761581421 and parameters: {'batch_size': 28, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:22:27,988] Trial 375 finished with value: 0.6468750238418579 and parameters: {'batch_size': 18, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:25:39,626] Trial 376 finished with value: 0.625 and parameters: {'batch_size': 28, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:26:48,373] Trial 374 finished with value: 0.6343749761581421 and parameters: {'batch_size': 24, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:31:31,834] Trial 377 finished with value: 0.6781250238418579 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:34:26,400] Trial 378 finished with value: 0.6187499761581421 and parameters: {'batch_size': 27, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:37:09,999] Trial 379 finished with value: 0.628125011920929 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:38:19,642] Trial 380 finished with value: 0.6625000238418579 and parameters: {'batch_size': 5, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:42:49,873] Trial 381 finished with value: 0.6875 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:43:55,742] Trial 384 finished with value: 0.6656249761581421 and parameters: {'batch_size': 26, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:45:43,730] Trial 382 finished with value: 0.6781250238418579 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:48:19,852] Trial 383 finished with value: 0.684374988079071 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:53:52,818] Trial 385 finished with value: 0.640625 and parameters: {'batch_size': 33, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:54:55,016] Trial 386 finished with value: 0.659375011920929 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:56:40,896] Trial 387 finished with value: 0.6156250238418579 and parameters: {'batch_size': 34, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 15:59:13,328] Trial 388 finished with value: 0.625 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:05:44,444] Trial 389 finished with value: 0.640625 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:06:55,429] Trial 390 finished with value: 0.612500011920929 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:08:55,666] Trial 391 finished with value: 0.675000011920929 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:11:58,898] Trial 392 finished with value: 0.606249988079071 and parameters: {'batch_size': 32, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:18:41,390] Trial 393 finished with value: 0.625 and parameters: {'batch_size': 34, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:19:49,163] Trial 394 finished with value: 0.668749988079071 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:21:53,534] Trial 395 finished with value: 0.671875 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:24:39,189] Trial 396 finished with value: 0.65625 and parameters: {'batch_size': 34, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:31:14,951] Trial 397 finished with value: 0.6499999761581421 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:32:19,035] Trial 398 finished with value: 0.609375 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:34:29,256] Trial 399 finished with value: 0.6468750238418579 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:37:19,148] Trial 400 finished with value: 0.6499999761581421 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:43:45,009] Trial 401 finished with value: 0.6312500238418579 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:44:34,205] Trial 402 finished with value: 0.6312500238418579 and parameters: {'batch_size': 31, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:45:18,145] Trial 406 finished with value: 0.637499988079071 and parameters: {'batch_size': 26, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:46:46,843] Trial 403 finished with value: 0.65625 and parameters: {'batch_size': 26, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:49:28,991] Trial 404 finished with value: 0.6312500238418579 and parameters: {'batch_size': 36, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:56:02,271] Trial 405 finished with value: 0.6343749761581421 and parameters: {'batch_size': 37, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:57:36,855] Trial 407 finished with value: 0.65625 and parameters: {'batch_size': 36, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 16:59:04,252] Trial 408 finished with value: 0.628125011920929 and parameters: {'batch_size': 26, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:01:36,332] Trial 409 finished with value: 0.659375011920929 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:09:03,941] Trial 410 finished with value: 0.6312500238418579 and parameters: {'batch_size': 27, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:10:45,612] Trial 411 finished with value: 0.6343749761581421 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:12:22,995] Trial 412 finished with value: 0.6499999761581421 and parameters: {'batch_size': 19, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:13:18,999] Trial 416 finished with value: 0.65625 and parameters: {'batch_size': 33, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:14:52,328] Trial 417 finished with value: 0.6312500238418579 and parameters: {'batch_size': 33, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:15:09,357] Trial 413 finished with value: 0.6468750238418579 and parameters: {'batch_size': 13, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:22:13,850] Trial 419 finished with value: 0.6468750238418579 and parameters: {'batch_size': 91, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:22:54,608] Trial 414 finished with value: 0.668749988079071 and parameters: {'batch_size': 33, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:23:43,006] Trial 420 finished with value: 0.625 and parameters: {'batch_size': 21, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:24:35,540] Trial 415 finished with value: 0.640625 and parameters: {'batch_size': 33, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:28:34,754] Trial 418 finished with value: 0.621874988079071 and parameters: {'batch_size': 24, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:30:34,274] Trial 422 finished with value: 0.6156250238418579 and parameters: {'batch_size': 10, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:35:54,409] Trial 421 finished with value: 0.6468750238418579 and parameters: {'batch_size': 24, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:37:17,575] Trial 423 finished with value: 0.653124988079071 and parameters: {'batch_size': 23, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:40:35,989] Trial 424 finished with value: 0.6625000238418579 and parameters: {'batch_size': 10, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:41:22,313] Trial 426 finished with value: 0.65625 and parameters: {'batch_size': 27, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:42:15,505] Trial 425 finished with value: 0.621874988079071 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:46:50,072] Trial 428 finished with value: 0.6468750238418579 and parameters: {'batch_size': 25, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:49:20,752] Trial 427 finished with value: 0.6625000238418579 and parameters: {'batch_size': 16, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:54:28,164] Trial 429 finished with value: 0.6312500238418579 and parameters: {'batch_size': 16, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:55:35,288] Trial 430 finished with value: 0.640625 and parameters: {'batch_size': 14, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 17:56:08,992] Trial 432 finished with value: 0.65625 and parameters: {'batch_size': 31, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:00:30,521] Trial 431 finished with value: 0.6499999761581421 and parameters: {'batch_size': 56, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:02:40,459] Trial 434 finished with value: 0.6499999761581421 and parameters: {'batch_size': 31, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:03:11,264] Trial 435 finished with value: 0.606249988079071 and parameters: {'batch_size': 20, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:04:42,966] Trial 438 finished with value: 0.6656249761581421 and parameters: {'batch_size': 15, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:08:26,147] Trial 433 finished with value: 0.6312500238418579 and parameters: {'batch_size': 30, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:09:47,212] Trial 437 finished with value: 0.675000011920929 and parameters: {'batch_size': 29, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:11:42,530] Trial 439 finished with value: 0.659375011920929 and parameters: {'batch_size': 27, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:14:29,469] Trial 442 finished with value: 0.643750011920929 and parameters: {'batch_size': 12, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:14:37,677] Trial 436 finished with value: 0.6499999761581421 and parameters: {'batch_size': 29, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:16:00,639] Trial 443 finished with value: 0.640625 and parameters: {'batch_size': 18, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:16:43,809] Trial 441 finished with value: 0.643750011920929 and parameters: {'batch_size': 100, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:21:39,763] Trial 444 finished with value: 0.612500011920929 and parameters: {'batch_size': 65, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:22:22,112] Trial 440 finished with value: 0.6499999761581421 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:29:16,710] Trial 448 finished with value: 0.625 and parameters: {'batch_size': 22, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:29:46,510] Trial 445 finished with value: 0.6499999761581421 and parameters: {'batch_size': 35, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:29:59,395] Trial 449 finished with value: 0.6312500238418579 and parameters: {'batch_size': 34, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:30:29,386] Trial 446 finished with value: 0.6625000238418579 and parameters: {'batch_size': 99, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:32:54,001] Trial 452 finished with value: 0.659375011920929 and parameters: {'batch_size': 25, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:34:30,628] Trial 447 finished with value: 0.640625 and parameters: {'batch_size': 92, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:35:34,293] Trial 450 finished with value: 0.6156250238418579 and parameters: {'batch_size': 27, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:35:47,362] Trial 451 finished with value: 0.637499988079071 and parameters: {'batch_size': 30, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:38:32,973] Trial 453 finished with value: 0.659375011920929 and parameters: {'batch_size': 91, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:40:16,002] Trial 454 finished with value: 0.659375011920929 and parameters: {'batch_size': 93, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:41:16,382] Trial 455 finished with value: 0.612500011920929 and parameters: {'batch_size': 94, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:41:27,610] Trial 456 finished with value: 0.653124988079071 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:44:10,651] Trial 457 finished with value: 0.6781250238418579 and parameters: {'batch_size': 25, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:45:56,332] Trial 458 finished with value: 0.6625000238418579 and parameters: {'batch_size': 96, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:47:01,544] Trial 459 finished with value: 0.653124988079071 and parameters: {'batch_size': 21, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:47:14,234] Trial 460 finished with value: 0.6343749761581421 and parameters: {'batch_size': 21, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:50:21,075] Trial 461 finished with value: 0.659375011920929 and parameters: {'batch_size': 98, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:52:18,545] Trial 462 finished with value: 0.659375011920929 and parameters: {'batch_size': 26, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:53:23,319] Trial 463 finished with value: 0.640625 and parameters: {'batch_size': 23, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:53:29,654] Trial 464 finished with value: 0.625 and parameters: {'batch_size': 15, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:55:48,408] Trial 467 finished with value: 0.6031249761581421 and parameters: {'batch_size': 19, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:55:54,619] Trial 468 finished with value: 0.640625 and parameters: {'batch_size': 30, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:56:27,611] Trial 465 finished with value: 0.6656249761581421 and parameters: {'batch_size': 32, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:57:07,486] Trial 469 finished with value: 0.637499988079071 and parameters: {'batch_size': 30, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:57:12,496] Trial 470 finished with value: 0.6000000238418579 and parameters: {'batch_size': 33, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:57:44,724] Trial 471 finished with value: 0.6656249761581421 and parameters: {'batch_size': 98, 'n_epochs': 100}. Best is trial 3 with value: 0.6875. [I 2021-03-11 18:58:16,575] Trial 466 finished with value: 0.6343749761581421 and parameters: {'batch_size': 32, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:03:02,487] Trial 472 finished with value: 0.637499988079071 and parameters: {'batch_size': 18, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:03:05,090] Trial 473 finished with value: 0.637499988079071 and parameters: {'batch_size': 98, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:04:07,226] Trial 475 finished with value: 0.628125011920929 and parameters: {'batch_size': 89, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:08:50,068] Trial 477 finished with value: 0.621874988079071 and parameters: {'batch_size': 25, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:08:53,010] Trial 476 finished with value: 0.637499988079071 and parameters: {'batch_size': 38, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:09:18,583] Trial 474 finished with value: 0.6781250238418579 and parameters: {'batch_size': 38, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:09:48,769] Trial 478 finished with value: 0.6468750238418579 and parameters: {'batch_size': 38, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:14:28,717] Trial 479 finished with value: 0.606249988079071 and parameters: {'batch_size': 20, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:14:32,556] Trial 480 finished with value: 0.621874988079071 and parameters: {'batch_size': 13, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:16:51,874] Trial 483 finished with value: 0.612500011920929 and parameters: {'batch_size': 40, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:16:56,871] Trial 484 finished with value: 0.6499999761581421 and parameters: {'batch_size': 41, 'n_epochs': 200}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:17:31,876] Trial 485 finished with value: 0.628125011920929 and parameters: {'batch_size': 41, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:17:37,712] Trial 486 finished with value: 0.65625 and parameters: {'batch_size': 96, 'n_epochs': 50}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:20:41,393] Trial 481 finished with value: 0.637499988079071 and parameters: {'batch_size': 14, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:21:06,576] Trial 482 finished with value: 0.625 and parameters: {'batch_size': 12, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:27:11,883] Trial 489 finished with value: 0.581250011920929 and parameters: {'batch_size': 35, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:27:42,544] Trial 490 finished with value: 0.6656249761581421 and parameters: {'batch_size': 24, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:30:20,007] Trial 487 finished with value: 0.625 and parameters: {'batch_size': 35, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:30:23,552] Trial 488 finished with value: 0.6468750238418579 and parameters: {'batch_size': 36, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:34:01,068] Trial 491 finished with value: 0.628125011920929 and parameters: {'batch_size': 29, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:41:19,206] Trial 492 finished with value: 0.6499999761581421 and parameters: {'batch_size': 43, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:43:53,161] Trial 494 finished with value: 0.671875 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:43:56,316] Trial 493 finished with value: 0.6625000238418579 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:47:26,297] Trial 495 finished with value: 0.653124988079071 and parameters: {'batch_size': 16, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:50:33,606] Trial 498 finished with value: 0.6468750238418579 and parameters: {'batch_size': 93, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:52:48,450] Trial 499 finished with value: 0.637499988079071 and parameters: {'batch_size': 27, 'n_epochs': 500}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:53:03,621] Trial 496 finished with value: 0.625 and parameters: {'batch_size': 28, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875. [I 2021-03-11 19:54:12,545] Trial 497 finished with value: 0.6499999761581421 and parameters: {'batch_size': 27, 'n_epochs': 1000}. Best is trial 3 with value: 0.6875.
final_best_params = study_final.best_params
final_best_params, study_final.best_value
({'batch_size': 88, 'n_epochs': 500}, 0.6875)
hp_score_board = hp_score_board.append({'Phase':'Final Tuning - Epcoh/Batch Size',
'Best Value':study_final.best_value,
'Hyperparameters':f'{study_final.best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.659375 | {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000} |
| 1 | Coarse Tuning | 0.681250 | {'drop_l1': 0.11769241744475195, 'drop_l2': 0.1203918426967813, 'drop_l3': 0.3079277845699331, 'drop_l4': 0.16249139281479208, 'drop_l5': 0.3503972703488872, 'drop_l6': 0.5575415438784195, 'activation': 'relu', 'kernel_init': 'lecun_uniform'} |
| 2 | Fine Tuning - Optimizer | 0.690625 | {'optimizer': 'Adam', 'learning_rate': 0.000609593339702459, 'epsilon': 4.226899030767284e-06, 'beta_1': 0.9537281643103777, 'beta_2': 0.9390376626945334} |
| 3 | Final Tuning - Epcoh/Batch Size | 0.687500 | {'batch_size': 88, 'n_epochs': 500} |
0.688 i.e decreased by 0.3% at 500 epochs and batch size of 88. Hence we may not pick these values and use default values of 10 and 100 for batch_size and epochs respectivelystudy_final.trials_dataframe()
| number | value | datetime_start | datetime_complete | duration | params_batch_size | params_n_epochs | state | |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.631250 | 2021-03-11 03:12:56.182586 | 2021-03-11 03:22:51.987761 | 0 days 00:09:55.805175 | 67 | 1000 | COMPLETE |
| 1 | 1 | 0.646875 | 2021-03-11 03:12:56.188977 | 2021-03-11 03:14:47.862718 | 0 days 00:01:51.673741 | 53 | 100 | COMPLETE |
| 2 | 2 | 0.646875 | 2021-03-11 03:12:56.194028 | 2021-03-11 03:16:05.335289 | 0 days 00:03:09.141261 | 82 | 200 | COMPLETE |
| 3 | 3 | 0.687500 | 2021-03-11 03:12:56.196922 | 2021-03-11 03:18:40.137341 | 0 days 00:05:43.940419 | 88 | 500 | COMPLETE |
| 4 | 4 | 0.659375 | 2021-03-11 03:14:47.870719 | 2021-03-11 03:32:31.025247 | 0 days 00:17:43.154528 | 76 | 2000 | COMPLETE |
| ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 495 | 495 | 0.653125 | 2021-03-11 19:34:01.086450 | 2021-03-11 19:47:26.297446 | 0 days 00:13:25.210996 | 16 | 1000 | COMPLETE |
| 496 | 496 | 0.625000 | 2021-03-11 19:41:19.213880 | 2021-03-11 19:53:03.621168 | 0 days 00:11:44.407288 | 28 | 1000 | COMPLETE |
| 497 | 497 | 0.650000 | 2021-03-11 19:43:53.168954 | 2021-03-11 19:54:12.544656 | 0 days 00:10:19.375702 | 27 | 1000 | COMPLETE |
| 498 | 498 | 0.646875 | 2021-03-11 19:43:56.320329 | 2021-03-11 19:50:33.606441 | 0 days 00:06:37.286112 | 93 | 500 | COMPLETE |
| 499 | 499 | 0.637500 | 2021-03-11 19:47:26.311565 | 2021-03-11 19:52:48.449927 | 0 days 00:05:22.138362 | 27 | 500 | COMPLETE |
500 rows × 8 columns
optuna.visualization.plot_parallel_coordinate(study_final)
optuna.visualization.plot_optimization_history(study_final)
optuna.visualization.plot_slice(study_final)
model_best_params = { **hp_best_params, **final_best_params}
model_best_params
{'num_layers': 6,
'arch': 'A D B',
'neurons_per_layer': [64, 256, 1000, 10, 1000, 1000],
'activation': 'relu',
'kernel_init': 'lecun_uniform',
'drop_rate_per_layer': [0.11769241744475195,
0.1203918426967813,
0.3079277845699331,
0.16249139281479208,
0.3503972703488872,
0.5575415438784195],
'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7ff289ca40a0>,
'batch_size': 88,
'n_epochs': 500}
model_best_params['batch_size'] = 30
model_best_params['n_epochs'] = 150
model_best_params
{'num_layers': 6,
'arch': 'A D B',
'neurons_per_layer': [64, 256, 1000, 10, 1000, 1000],
'activation': 'relu',
'kernel_init': 'lecun_uniform',
'drop_rate_per_layer': [0.11769241744475195,
0.1203918426967813,
0.3079277845699331,
0.16249139281479208,
0.3503972703488872,
0.5575415438784195],
'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam at 0x7ff289ca40a0>,
'batch_size': 30,
'n_epochs': 150}
dnn_final = DNN_Model('classification')
m_final = dnn_final.create_model(X_train.shape[1],n_class,**model_best_params)
callbacks = LivePlot(5,train_loss='loss',train_metric='accuracy')
# callbacks=[]
dnn_final.train_model(m_final,X_train,y_train,['sparse_categorical_crossentropy'],['accuracy'],callbacks=[callbacks])
result = dnn_final.test_model(X_test,y_test)
display(Markdown("### Model Summary"))
print(m_final.summary())
display(Markdown(f"### Test Loss : {result[0]} Test Accuracy : {result[1]}"))
Model: "model_16" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_17 (InputLayer) [(None, 4)] 0 _________________________________________________________________ dense_53 (Dense) (None, 64) 320 _________________________________________________________________ activation_37 (Activation) (None, 64) 0 _________________________________________________________________ dropout_37 (Dropout) (None, 64) 0 _________________________________________________________________ batch_normalization_37 (Batc (None, 64) 192 _________________________________________________________________ dense_54 (Dense) (None, 256) 16640 _________________________________________________________________ activation_38 (Activation) (None, 256) 0 _________________________________________________________________ dropout_38 (Dropout) (None, 256) 0 _________________________________________________________________ batch_normalization_38 (Batc (None, 256) 768 _________________________________________________________________ dense_55 (Dense) (None, 1000) 257000 _________________________________________________________________ activation_39 (Activation) (None, 1000) 0 _________________________________________________________________ dropout_39 (Dropout) (None, 1000) 0 _________________________________________________________________ batch_normalization_39 (Batc (None, 1000) 3000 _________________________________________________________________ dense_56 (Dense) (None, 10) 10010 _________________________________________________________________ activation_40 (Activation) (None, 10) 0 _________________________________________________________________ dropout_40 (Dropout) (None, 10) 0 _________________________________________________________________ batch_normalization_40 (Batc (None, 10) 30 _________________________________________________________________ dense_57 (Dense) (None, 1000) 11000 _________________________________________________________________ activation_41 (Activation) (None, 1000) 0 _________________________________________________________________ dropout_41 (Dropout) (None, 1000) 0 _________________________________________________________________ batch_normalization_41 (Batc (None, 1000) 3000 _________________________________________________________________ dense_58 (Dense) (None, 1000) 1001000 _________________________________________________________________ activation_42 (Activation) (None, 1000) 0 _________________________________________________________________ dropout_42 (Dropout) (None, 1000) 0 _________________________________________________________________ batch_normalization_42 (Batc (None, 1000) 3000 _________________________________________________________________ dense_59 (Dense) (None, 6) 6006 ================================================================= Total params: 1,311,966 Trainable params: 1,305,306 Non-trainable params: 6,660 _________________________________________________________________ None
hp_score_board = hp_score_board.append({'Phase':'Model Testing - Train/Test',
'Best Value':f'{loss}: {result[0]} - {metric}: {result[1]}',
'Hyperparameters':f'{hp_best_params}'},ignore_index=True)
hp_score_board
| Phase | Best Value | Hyperparameters | |
|---|---|---|---|
| 0 | Architecture Selection | 0.659375 | {'num_layers': 6, 'arch': 'A D B', 'neuron_l1': 64, 'neuron_l2': 256, 'neuron_l3': 1000, 'neuron_l4': 10, 'neuron_l5': 1000, 'neuron_l6': 1000} |
| 1 | Coarse Tuning | 0.68125 | {'drop_l1': 0.11769241744475195, 'drop_l2': 0.1203918426967813, 'drop_l3': 0.3079277845699331, 'drop_l4': 0.16249139281479208, 'drop_l5': 0.3503972703488872, 'drop_l6': 0.5575415438784195, 'activation': 'relu', 'kernel_init': 'lecun_uniform'} |
| 2 | Fine Tuning - Optimizer | 0.690625 | {'optimizer': 'Adam', 'learning_rate': 0.000609593339702459, 'epsilon': 4.226899030767284e-06, 'beta_1': 0.9537281643103777, 'beta_2': 0.9390376626945334} |
| 3 | Final Tuning - Epcoh/Batch Size | 0.6875 | {'batch_size': 88, 'n_epochs': 500} |
| 4 | Model Testing - Train/Test | sparse_categorical_crossentropy: 0.913973331451416 - accuracy: 0.6499999761581421 | {'num_layers': 6, 'arch': 'A D B', 'neurons_per_layer': [64, 256, 1000, 10, 1000, 1000], 'activation': 'relu', 'kernel_init': 'lecun_uniform', 'drop_rate_per_layer': [0.11769241744475195, 0.1203918426967813, 0.3079277845699331, 0.16249139281479208, 0.3503972703488872, 0.5575415438784195], 'opts': <tensorflow.python.keras.optimizer_v2.adam.Adam object at 0x7ff289ca40a0>, 'batch_size': 88, 'n_epochs': 500} |
# opt_conf
temp_bp = model_best_params
temp_bp['opts'] = opt_conf
temp_bp
{'num_layers': 6,
'arch': 'A D B',
'neurons_per_layer': [64, 256, 1000, 10, 1000, 1000],
'activation': 'relu',
'kernel_init': 'lecun_uniform',
'drop_rate_per_layer': [0.11769241744475195,
0.1203918426967813,
0.3079277845699331,
0.16249139281479208,
0.3503972703488872,
0.5575415438784195],
'opts': {'optimizer': 'Adam',
'learning_rate': 0.000609593339702459,
'epsilon': 4.226899030767284e-06,
'beta_1': 0.9537281643103777,
'beta_2': 0.9390376626945334},
'batch_size': 30,
'n_epochs': 150}
Model Based Feature Selection was better than the other two.Sparse Categorical Entropy is used as a loss function and Accuracy as the performance metric in training and testing the ANN.A multi stage strategy was applied in tuning hyper parameters of the ANN.
Stage 1: Architecture Selection - Accuracy of 66%
Stage 2: Coarse tuning - Accuracy of 68%
Stage 3 : Fine tuning - Accuracy of 69%
Stage 4 : Final tuning - Accuracy of ~69%
However, all hyperparameters prior to final tuning were retained and manually tweaked for stage 4 parameters.
Streamlit an opensource is created to test the hyperparameters and to save the model. You can try the same @ http://34.70.58.49:8085/. Please make sure to upload the data file and copy paste the hyperparameters above in the respective text input area.